Skip to content

[Feature] Add GVA support for Lightning #81

Open
Open
[Feature] Add GVA support for Lightning#81
Assignees
sjmshsh
Labels
enhancementNew feature or requestgood first issueGood for newcomershelp wantedExtra attention is needed
@icavan

Description

@icavan
icavan
opened on May 25, 2026

Similar to #55

  • Support HV > H (num_v_heads > num_qk_heads) in KDA, following the gated_delta_rule GVA pattern
  • Add corresponding test and benchmark config

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestgood first issueGood for newcomershelp wantedExtra attention is needed

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions