|
408 tags across 66 posts
Tags by category
Transformer System 408 tags
numinens 66 transformer 62 kv-cache 21 deployment 13 inference 11 quantization 11 decode 10 serving 9 training 9 speculative-decoding 8 tensor-shapes 6 attention 5 lora 5 multimodal 5 prefill 5 scheduling 5 vllm 5 kernel 4 moe 4 parallel-decoding 4 rope 4 routing 4 softmax 4 continuous-batching 3 deepseek 3 fine-tuning 3 fp8 3 goodput 3 int4 3 kernels 3 knowledge-distillation 3 long-context 3 low-rank 3 memory 3 peft 3 ptq 3 qkv 3 scaling-laws 3 tree-attention 3 x1 3 adapter-serving 2 all-to-all 2 beam-search 2 calibration 2 capacity 2 clipping 2 context-geometry 2 cross-attention 2 encoder 2 expert-parallelism 2 experts 2 extrapolation 2 flash-decoding 2 flashattention 2 flashattention-3 2 gpu 2 gqa 2 hbm 2 hessian 2 inference-kernels 2 int8 2 lifecycle-cost 2 logits 2 loss 2 medusa 2 mixed-precision 2 model-compression 2 mtp 2 multi-token-prediction 2 ntk 2 reproducibility 2 retrieval 2 rollback 2 sampling 2 sdpa 2 straight-through-estimator 2 teacher-forcing 2 tiling 2 top-k 2 tpot 2 ttft 2 vocabulary 2 weight-only 2 weight-tying 2 accept-reject 1 action-tokenization 1 activation 1 activation-quantization 1 activations 1 adalora 1 adam 1 additive-quantization 1 affine-quantization 1 agent-engineering 1 alibi 1 aqlm 1 architecture 1 attention-bias 1 attention-mask 1 audio 1 autocompressor 1 autoregressive 1 awq 1 bandwidth 1 batching 1 benchmarking 1 bert 1 bi-encoder 1 binary-weights 1 bitlinear 1 bitnet 1 bitnet-b1-58 1 blackwell 1 blip-2 1 block-assembly 1 bpe 1 canary 1 catastrophic-forgetting 1 causal-mask 1 checkpoint-sharding 1 checkpointing 1 chinchilla 1 cholesky 1 classification 1 clover 1 cnn 1 codebook 1 compute-optimal 1 conditional-computation 1 conditional-compute 1 constrained-reinforcement-learning 1 context-window 1 continual-learning 1 cost-model 1 cpu-memory 1 cross-encoder 1 cuda 1 data 1 data-constraints 1 data-pipeline 1 decoder 1 decoding 1 decoding-policy 1 deepep 1 deepnorm 1 deepseek-v3 1 disaggregation 1 dispatch 1 distributed-systems 1 distserve 1 document-ai 1 dora 1 downstream-evaluation 1 draft-model 1 dropout 1 duquant 1 dynamic-quantization 1 eagle 1 eagle-2 1 eagle-3 1 effective-data 1 efficientqat 1 embeddings 1 emergence 1 energy 1 eplb 1 equivalent-transform 1 evaluation 1 evidence 1 execution 1 experiments 1 extreme-compression 1 fake-quantization 1 feature-drafting 1 feed-forward-network 1 ffn 1 flamingo 1 flashattention-2 1 flashattention-4 1 flashmask 1 flashmla 1 flatquant 1 flops 1 foundation-models 1 fusion 1 gated-mlp 1 gelu 1 gemm 1 geometry 1 gguf 1 glu 1 gptq 1 gradients 1 grouped-gemm 1 h2o 1 hadamard 1 hass 1 head-layout 1 hierarchical-memory 1 hopper 1 hydra 1 hydralora 1 incoherence 1 infinigen 1 initial-state 1 intrinsic-dimension 1 io-aware 1 jacobi-decoding 1 kivi 1 kvquant 1 label-smoothing 1 late-interaction 1 latency 1 lattice 1 layernorm 1 learning 1 length-extrapolation 1 linear-algebra 1 llama 1 llama-cpp 1 llava 1 llm-int8 1 llm-qat 1 llm-serving 1 lm-head 1 lmcache 1 load-balancing 1 loftq 1 log-softmax 1 longrope 1 lookahead-decoding 1 lookup-table 1 lora-fa 1 lora-plus 1 loss-mask 1 low-bit-training 1 lr-qat 1 lsq 1 mask 1 massive-activations 1 measurement 1 megablocks 1 memory-management 1 memserve 1 mha 1 mixtral 1 mixture-of-attention 1 mixture-of-depths 1 mixture-of-experts 1 mixture-of-loras 1 mla 1 mlp 1 model-parallelism 1 model-selection 1 model-serialization 1 mooncake 1 mqa 1 multi-head-attention 1 n-gram 1 nvme 1 obd 1 obq 1 obs 1 observability 1 observers 1 occupancy 1 offload 1 omniquant 1 on-policy-distillation 1 onebit 1 onnx 1 optimization 1 optimizer 1 orca 1 outliers 1 output-head 1 over-training 1 packed-sequence 1 pact 1 padding-mask 1 paged-attention 1 pagedattention 1 paging 1 parallelism 1 pissa 1 pooling 1 position-encoding 1 position-ids 1 post-ln 1 pre-ln 1 prediction-heads 1 preface 1 prefetch 1 prefix-caching 1 prefixquant 1 profiling 1 projection 1 projector 1 pruning 1 publication 1 punica 1 pyramidinfer 1 pyramidkv 1 q-former 1 qat 1 qlora 1 qserve 1 quality-gates 1 quantization-aware-training 1 quarot 1 question-answering 1 quip-sharp 1 radixattention 1 reinforcement-learning 1 relative-position 1 release-engineering 1 representation 1 resampler 1 research-method 1 residual-connection 1 residual-networks 1 residual-stream 1 rest 1 retention 1 retrievalattention 1 rmsnorm 1 rnn 1 roadmap 1 roofline 1 rotary-position-embedding 1 rounding 1 rslora 1 rtn 1 runtime 1 runtime-integration 1 s-lora 1 safetensors 1 sarathi 1 scaled-dot-product-attention 1 scales 1 scheduler 1 self-attention 1 sensitivity 1 sentence-embedding 1 seq2seq 1 sequence-contract 1 sequence-distillation 1 sequoia 1 shared-memory 1 sinusoidal 1 smoothattention 1 smoothquant 1 snapkv 1 sparse-kernels 1 sparse-quantization 1 sparsification 1 sparsity 1 specinfer 1 spectr 1 speculative-sampling 1 spinquant 1 splitwise 1 spqr 1 squeezellm 1 sram 1 state-machine 1 stopping-criteria 1 streaming 1 streamingllm 1 structural-compression 1 structured-pruning 1 student-model 1 subspace 1 subwords 1 svd 1 swiglu 1 switchhead 1 t5 1 task-head 1 teacher-student 1 technical-writing 1 temperature 1 tensorrt-llm 1 ternary-weights 1 tma 1 tmem 1 token-classification 1 tokenization 1 tokens 1 top-p 1 torchao 1 training-runtime 1 training-stability 1 transformer-xl 1 unitization 1 unstructured-pruning 1 vattention 1 vector-quantization 1 video 1 vision-encoder 1 vision-transformer 1 visualization 1 vptq 1 w4a16 1 wgmma 1 word2vec 1 yarn 1 zero-point 1