Featurization

Featurization methods are used to transform textual attributes of entities (e.g. file paths, process command lines, socket IP addresses and ports) into a vector. Some methods like word2vec and doc2vec learn this vector from the text corpus, while others like hierarchical_hashing compute the vector in a deterministic way.

Other methods like only_type and only_ones simply skip this embedding step and assign either a one-hot encoded type or ones to each entity. Those methods thus do not require any specific argument. In all methods, the resulting vectors are used as node features during the training task.

The pretrained method uses an encoder pretrained on provenance data (SPIDER, language models, self-supervised GNNs, …), selected with pretrained.model_type. See Pretrained Encoders.

• word2vec
  • alpha: float
  • window_size: int
  • min_count: int
  • use_skip_gram: bool
  • num_workers: int
  • epochs: int
  • compute_loss: bool
  • negative: int
  • decline_rate: int
• doc2vec
  • include_neighbors: bool
  • epochs: int
  • alpha: float
• fasttext
  • min_count: int
  • alpha: float
  • window_size: int
  • negative: int
  • num_workers: int
  • use_pretrained_fb_model: bool
• alacarte
  • walk_length: int
  • num_walks: int
  • epochs: int
  • context_window_size: int
  • min_count: int
  • use_skip_gram: bool
  • num_workers: int
  • compute_loss: bool
  • add_paths: bool
• temporal_rw
  • walk_length: int
  • num_walks: int
  • trw_workers: int
  • time_weight: str
  • half_life: int
  • window_size: int
  • min_count: int
  • use_skip_gram: bool
  • wv_workers: int
  • epochs: int
  • compute_loss: bool
  • negative: int
  • decline_rate: int
• flash
  • min_count: int
  • workers: int
• hierarchical_hashing
• magic
• only_type
• only_ones
• ocrapt_features
  • use_lifespan: bool (1)
  • use_cumulative_active_time: bool (2)
• pretrained
  • model_size: str (3)
  • model_type: str (4)
  • deepwalk
    • window: int (5)
    • epochs: int (6)
    • min_count: int (7)
    • workers: int (8)
  • node2vec
    • p: float (9)
    • q: float (10)
  • gnn_distill
    • emb_dim: int (11)
    • hidden_dim: int (12)
    • num_layers: int (13)
    • num_heads: int (14)
    • n_neighbors_min: int (15)
    • n_neighbors_max: int (16)
    • diverse_neighbors: bool (17)
    • loss_weight: float (18)
    • gnn_loss_weight: float (19)
    • gnn_lr: float (20)
    • ema_momentum: float (21)
    • edge_projection: bool (22)
    • filter_noisy_edges: bool (23)
  • behavior_cluster
    • proj_dim: int (24)
    • bce_weight: float (25)
    • contrastive_weight: float (26)
    • temperature: float (27)
    • min_signature_size: int (28)
    • filter_noisy_edges: bool (29)
    • use_contrastive_head: bool (30)
    • min_entities_per_class: int (31)
    • max_entities_per_class: int (32)
    • samples_per_class: int (33)
    • bce_mode: str (34)
    • contrastive_target: str (35)
    • strip_entity_type: bool (36)
  • spider
    • emb_dim: int (37)
    • hidden_dim: int (38)
    • proj_dim: int (39)
    • num_heads: int (40)
    • n_neighbors_min: int (41)
    • n_neighbors_max: int (42)
    • diverse_neighbors: bool (43)
    • filter_noisy_edges: bool (44)
    • ema_momentum: float (45)
    • supcon_weight: float (46)
    • distill_weight: float (47)
    • temperature: float (48)
    • min_signature_size: int (49)
    • min_entities_per_class: int (50)
    • max_entities_per_class: int (51)
    • samples_per_class: int (52)
    • strip_entity_type: bool (53)
    • distill_loss: str (54)
    • teacher_loss: str (55)
    • teacher_data: str (56)
    • student_only_mode: str (57)
  • gpt2_pretrained
    • max_seq_len: int (58)
  • llama3_pretrained
    • max_seq_len: int (59)
  • opt_pretrained
    • max_seq_len: int (60)
  • graphmae
    • num_layers: int (61)
    • num_heads: int (62)
    • decoder_num_layers: int (63)
    • mask_rate: float (64)
    • replace_rate: float (65)
    • neighborhood_min: int (66)
    • neighborhood_max: int (67)
    • epochs: int (68)
    • lr: float (69)
  • gae
    • num_layers: int (70)
    • num_heads: int (71)
    • neighborhood_min: int (72)
    • neighborhood_max: int (73)
    • epochs: int (74)
    • lr: float (75)
  • dgi
    • num_layers: int (76)
    • num_heads: int (77)
    • neighborhood_min: int (78)
    • neighborhood_max: int (79)
    • epochs: int (80)
    • lr: float (81)
  • training
    • pretrain_tokens: int (82)
    • warmup_tokens: int (83)
    • batch_size: int (84)
    • lr: float (85)
    • scheduler: str (86)
  • mlm
    • mask_rate_fixed: float (87)
    • mask_rate_min: float (88)
    • mask_edge_type: bool (89)
    • modernbert
      • global_attn_every_n_layers: int (90)
      • local_attention_window: int (91)
    • ropebert
      • rope_theta: float (92)
    • llama
      • rope_theta: float (93)
    • logbert
      • hvm_weight: float (94)
  • pretrain_datasets: str (95)
  • tokenizer
    • bpe_vocab_size: int (96)
    • max_seq_len: int (97)
    • mode: str (98)
    • normalize_netflow_ips: bool (99)
    • canonicalize_neighbors: bool (100)
  • weights_path: str (101)
  • graph_context_mode: str (102)
  • walks
    • walk_length: int (103)
    • num_walks: int (104)
    • time_weight: str (105)
    • half_life: float (106)
    • random_walk_start: bool (107)
    • diversity_weight: float (108)

1. Off by default, hurts generalization (paper Appendix E).
   
2. Off by default, same reason as use_lifespan.
   
3. SPIDER encoder size preset (selects hidden / layers / heads / FFN). Standard presets: 'tiny' (H=128), 'mini' (H=256), 'med' (H=512), 'baseline' (H=768). HF pretrained model_types use their own size keys (e.g. 'small' / 'medium' / 'large' / 'xl' for gpt2_pretrained, '1b' / '3b' for llama3_pretrained).
   
4. SPIDER pretraining objective + architecture. See config/pretrained/README.md for full descriptions. Families: MLM walk-based : bert, roberta, modernbert, ropebert, llama, logbert Walk embedding : deepwalk, node2vec GNN self-supervised: graphmae, gae, dgi GNN token-budget : gnn_distill, behavior_cluster, spider HF pretrained : gpt2_pretrained, llama3_pretrained, opt_pretrained
   
   Available options (one selection):
   bert
roberta
modernbert
ropebert
llama
logbert
deepwalk
node2vec
graphmae
gae
dgi
gnn_distill
behavior_cluster
spider
gpt2_pretrained
llama3_pretrained
opt_pretrained
5. Skip-gram context window size. Default: 5.
   
6. Number of Word2Vec training epochs. Default: 10.
   
7. Minimum label frequency to include in vocabulary. Default: 1.
   
8. Number of parallel workers for Word2Vec training. Default: 4.
   
9. Return parameter. Higher p = less likely to revisit previous node. Default: 1.0.
   
10. In-out parameter. q > 1 = BFS-like (local); q < 1 = DFS-like (explore). Default: 1.0.
    
11. GNN edge embedding dimension. Default: 128.
    
12. GNN hidden dimension. Default: 128.
    
13. Number of GNN encoder layers. Default: 1.
    
14. Number of attention heads in GNN layers. Default: 4.
    
15. Min direct neighbors to sample per entity. Default: 5.
    
16. Max direct neighbors to sample per entity. Default: 20.
    
17. Dedup neighbors by (edge_type, node_type, label) and prioritize edge type diversity. Default: True.
    
18. Weight for distillation loss (T5 encoder → GNN alignment). Default: 0.5.
    
19. Weight for GNN masked reconstruction loss. Default: 1.0.
    
20. Learning rate for GNN teacher parameters. Default: 1e-3.
    
21. EMA momentum for teacher update (0.999 = slow-moving). Default: 0.999.
    
22. Apply per-edge-type linear projection to source nodes before GNN message passing. Default: False.
    
23. Remove shared libs, /dev, /proc, /sys, linker cache, common configs from GNN neighborhoods. Default: False.
    
24. Contrastive projection head output dimensionality. Default: 128.
    
25. Weight for multi-label BCE loss. Default: 1.0.
    
26. Weight for contrastive (NT-Xent) loss. Default: 0.5.
    
27. Temperature for InfoNCE cosine similarity scaling. Default: 0.07.
    
28. Skip entities with fewer behavior labels in their signature. Default: 2.
    
29. Remove shared libs, /dev, /proc, /sys, linker cache, common configs from signature extraction. Default: True.
    
30. Use contrastive projection head for inference embeddings (proj_dim) instead of raw encoder (emb_dim). Default: False.
    
31. Floor: oversample small classes to this minimum per epoch. 0 = no floor. Default: 0.
    
32. Cap entities per signature class per epoch. Rotates across epochs. 0 = no cap. Default: 0.
    
33. K in P×K batch sampling: number of entities per class per batch. Guarantees every entity has at least K-1 positives for contrastive learning. Default: 4.
    
34. Classification head mode: 'multilabel' = BCE over behavior labels, 'class' = cross-entropy over entity classes. Default: multilabel.
    
35. Contrastive positive definition: 'signature' = entities with identical behavior signatures are positives, 'entity_class' = entities with the same coarse functional class are positives. Default: signature.
    
36. Remove entity type prefix tokens ([PROC], [FILE], [SOCK]) from tokenized sequences. Forces the model to learn identity from behavior rather than type. Default: False.
    
37. GNN edge embedding dimension. Default: 256.
    
38. GNN encoder hidden dimension. Default: 256.
    
39. Contrastive projection dimension (teacher). Default: 256.
    
40. Number of attention heads in GNN TransformerConv. Default: 4.
    
41. Min direct neighbors to sample per entity. Default: 5.
    
42. Max direct neighbors to sample per entity. Default: 20.
    
43. Dedup neighbors by (edge_type, node_type, label) and prioritize edge type diversity. Default: True.
    
44. Remove shared libs, /dev, /proc, /sys from GNN neighborhoods. Default: True.
    
45. EMA momentum for teacher T5 update. Default: 0.999.
    
46. Weight for supervised contrastive loss on GNN teacher. Default: 1.0.
    
47. Weight for distillation loss (student → GNN targets). Default: 0.5.
    
48. SupCon temperature. Default: 0.07.
    
49. Skip entities with fewer behavior labels (for entity class assignment). Default: 2.
    
50. Floor: oversample small classes to this minimum per epoch. Default: 100.
    
51. Cap entities per class per epoch. Default: 2000.
    
52. K in P×K batching. Default: 4.
    
53. Remove entity type prefix tokens from inputs. Default: False.
    
54. Distillation loss: sce or mse. Default: sce.
    
    Available options (one selection):
    sce
mse
55. Teacher loss: contrastive (SupCon) or bce (cross-entropy). Default: contrastive.
    
    Available options (one selection):
    contrastive
bce
56. Teacher data modalities (comma-separated): signature, gnn_emb, or both. Default: signature,gnn_emb.
    
57. Student-only ablation: none, student_signature, or student_class. Default: none.
    
    Available options (one selection):
    none
student_signature
student_class
58. Max sequence length for HF tokenizer. Default: 512.
    
59. Max sequence length for HF tokenizer. Default: 512.
    
60. Max sequence length for HF tokenizer. Default: 512.
    
61. Number of GAT encoder layers. Default: 2.
    
62. Number of attention heads in GAT layers. Default: 4.
    
63. Number of GAT decoder layers. Default: 1.
    
64. Fraction of nodes to mask during pretraining. Default: 0.5.
    
65. Fraction of masked nodes replaced with random tokens (rest get [MASK]). Default: 0.1.
    
66. Minimum number of neighbors in temporal neighborhood. Default: 5.
    
67. Maximum number of neighbors in temporal neighborhood. Default: 20.
    
68. Number of training epochs. Default: 100.
    
69. Learning rate. Default: 0.001.
    
70. Number of GAT encoder layers. Default: 2.
    
71. Number of attention heads in GAT layers. Default: 4.
    
72. Minimum number of neighbors in temporal neighborhood. Default: 5.
    
73. Maximum number of neighbors in temporal neighborhood. Default: 20.
    
74. Number of training epochs. Default: 100.
    
75. Learning rate. Default: 0.001.
    
76. Number of GAT encoder layers. Default: 2.
    
77. Number of attention heads in GAT layers. Default: 4.
    
78. Minimum number of neighbors in temporal neighborhood. Default: 5.
    
79. Maximum number of neighbors in temporal neighborhood. Default: 20.
    
80. Number of training epochs. Default: 100.
    
81. Learning rate. Default: 0.001.
    
82. Total number of tokens to process during pretraining (controls training duration).
    
83. Number of tokens for learning rate warmup at the start of pretraining.
    
84. Mini-batch size during pretraining. Interpretation depends on model_type: walks per batch (MLM), entities per batch (GNN token-budget), labels per batch (HF pretrained), neighborhoods per batch (GNN-SSL).
    
85. Peak learning rate for pretraining (after warmup).
    
86. Learning rate scheduler shape after warmup. Only consumed by MLM and HF pretrained; GNN token-budget hardcodes cosine.
    
    Available options (one selection):
    cosine
linear
87. Initial fixed mask rate (fraction of nodes masked). Decays to mask_rate_min.
    
88. Minimum mask rate after decay.
    
89. Enable structured masking of edge types during pretraining and edge-type scoring at inference.
    
90. Use global attention every N layers. Other layers use local sliding window attention. Default: 3.
    
91. Sliding window size for local attention layers. Default: 128 tokens.
    
92. Base frequency for rotary position embeddings. Default: 10000.0.
    
93. Base frequency for rotary position embeddings. Default: 10000.0.
    
94. Weight for hypersphere volume minimization loss relative to MLM loss. Default: 0.1.
    
95. Comma-separated list of dataset names to use for multi-dataset pretraining.
    
96. Target BPE vocabulary size for the tokenizer.
    
97. Maximum token sequence length after tokenization. Walks exceeding this are truncated.
    
98. Tokenizer mode: 'domain_bpe' uses domain-specific pre-tokenization followed by BPE; 'bpe_only' skips domain rules and applies BPE directly on raw words.
    
    Available options (one selection):
    domain_bpe
bpe_only
99. Replace IP addresses in netflow entities with category tokens ([PRIVATE_IP], [PUBLIC_IP], [LOCALHOST_IP]) instead of keeping individual octets.
    
100. Enable multi-level neighbor canonicalization. When True, decoder targets keep only structural/categorical tokens ([] special tokens + OS-agnostic category tokens like [CAT_WEBSERVER], [FCAT_LOG_WEB]) while encoder input gets full detail + category tokens prepended. During continue_pretrain, neighbors keep all tokens with category tokens prepended.
     
101. Path to a folder of pretrained weights (e.g. the SPIDER weights). May contain any subset of: corpus.pt (sampler states + indexid2msg), tokenizer.pt, pretrain_*.pt (model checkpoint), behavior_vocab.txt. Present artifacts are loaded; missing ones are computed from scratch. Works with any model_type.
     
102. Graph context scope for walk sampling at both pretraining and inference time. 'window' = each time-window snapshot independently; 'day' = merge all snapshots from the same calendar day; 'all' = merge all snapshots in the relevant split (train split at pretrain time; val or test split at inference time — no training data leaks into inference context).
     
     Available options (one selection):
     window
day
all
103. Number of nodes per random walk during pretraining.
     
104. Number of random walks to sample per node per epoch during pretraining.
     
105. Temporal weighting for neighbor selection: 'uniform', 'exponential', or 'linear'.
     
106. Half-life (in seconds) for exponential time weighting of neighbor selection.
     
107. Randomize the temporal entry point for each walk. When True, the first hop picks a random edge instead of always starting from the earliest/latest timestamp.
     
108. Edge-type diversity bias for random walks during pretraining. 0 = no bias (default), higher values increasingly favor edges whose type is underrepresented in the current walk. Only affects pretraining; finetuning/inference use natural distribution.