Skip to content

Add Ovis2 model and processor implementation #37088

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Cyrilvallez merged 110 commits into from
Aug 18, 2025
Merged

Add Ovis2 model and processor implementation #37088

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
b3bfa35
Add Ovis2 model and processor implementation
thisisiron Mar 28, 2025
51c9efd
Apply style fixes
thisisiron Mar 28, 2025
9891508
Add unit tests for Ovis2 image processing and processor
thisisiron Mar 29, 2025
fde1b2a
Refactor image processing functions for clarity and efficiency
thisisiron Mar 29, 2025
6b0e5d4
Add Ovis2 ImageProcessorFast
thisisiron Mar 30, 2025
6b8ae7e
Refactor Ovis2 code
thisisiron Mar 31, 2025
91f72b2
Refactor Ovis2 model components and update processor functionality
thisisiron Mar 31, 2025
aacbab3
Fix repo consistency issues for Ovis2: docstring, config cleanup
thisisiron Mar 31, 2025
7305a22
Update Ovis2 model integration tests
thisisiron Mar 31, 2025
355a91c
Update Ovis2 configuration and processing classes for improved docume…
thisisiron Mar 31, 2025
ac232e0
Remove duplicate entry for 'ovis2' in VLM_CLASS_NAMES
thisisiron Apr 1, 2025
16d71f8
Fix conflict
thisisiron Apr 1, 2025
a7b5094
Fix import order
thisisiron Apr 1, 2025
4d56043
Update image processor class names
thisisiron Apr 4, 2025
7f1cbc0
Update Ovis2 model structure
thisisiron May 5, 2025
a4e37e6
Refactor Ovis2 configuration
thisisiron May 5, 2025
11a2a09
Merge remote-tracking branch 'upstream/main' into add-ovis2
thisisiron May 5, 2025
5999659
Fix typos
thisisiron May 5, 2025
f66426c
Refactor Ovis2 model classes and remove unused code
thisisiron May 5, 2025
ae1ea0d
Fix typos
thisisiron May 5, 2025
4e540b5
Refactor Ovis2 model initialization
thisisiron May 5, 2025
83a7cca
Fiix typos
thisisiron May 5, 2025
234edb2
Merge branch 'main' into add-ovis2
thisisiron May 29, 2025
db59777
Remove Ovis2 model mapping from MODEL_MAPPING_NAMES in modeling_auto.py
thisisiron May 29, 2025
b604a70
Add license and update type hints
thisisiron May 29, 2025
f26717d
Refactor token function and update docstring handling
thisisiron May 29, 2025
890abdc
Add license
thisisiron May 30, 2025
97e84a4
Merge branch 'main' into add-ovis2
thisisiron May 30, 2025
67a45ab
Merge branch 'main' into add-ovis2
thisisiron May 31, 2025
764e74f
Merge branch 'main' into add-ovis2
thisisiron Jun 2, 2025
178fc10
Add Ovis2 model support and update documentation
thisisiron Jun 23, 2025
2e278a4
Refactor Ovis2 model structure and enhance multimodal capabilities
thisisiron Jun 23, 2025
17afef9
Update Ovis2 weight mapping for consistency and clarity in key patterns
thisisiron Jun 23, 2025
1a87ab3
Remove unused 'grids' parameter from Ovis2 model and Update processin…
thisisiron Jun 23, 2025
f3c498e
Refactor Ovis2 model test structure to include Ovis2Model
thisisiron Jun 23, 2025
ec0ffd5
Merge branch 'main' into add-ovis2
thisisiron Jun 23, 2025
0f418e8
Add optional disable_grouping param to Ovis2ImageProcessorFast
thisisiron Jun 23, 2025
afd50aa
Refactor type hints in Ovis2 modules
thisisiron Jun 23, 2025
bdbcb22
Add licensing information in Ovis2 modules and tests
thisisiron Jun 25, 2025
cd369a6
Refactor Ovis2 model by removing unused methods
thisisiron Jun 25, 2025
b459f50
Refactor Ovis2 model tests by renaming test classes and removing skip…
thisisiron Jun 25, 2025
4ae2f70
Merge branch 'main' into add-ovis2
thisisiron Jun 25, 2025
57abe35
Refactor Ovis2 model output classes
thisisiron Jun 25, 2025
541dc7f
Refactor Ovis2 weight conversion and Update model embedding classes
thisisiron Jun 28, 2025
5e7846c
Merge branch 'main' into add-ovis2
thisisiron Jun 28, 2025
d13eaea
Refactor Ovis2 model imports and remove unused functions
thisisiron Jun 28, 2025
a10e3db
Enhance vision configuration extraction in Ovis2 weight conversion
thisisiron Jun 28, 2025
0501e0f
Refactor Ovis2 model's forward method to remove interpolation option
thisisiron Jun 28, 2025
c19231f
Update Ovis2 model documentation
thisisiron Jun 28, 2025
6083141
Merge branch 'main' into add-ovis2
thisisiron Jul 3, 2025
c27bf25
Refactor Ovis2 model input handling and tokenizer configuration
thisisiron Jul 4, 2025
58c0c0a
Merge branch 'main' into add-ovis2
thisisiron Jul 4, 2025
94fd529
Update return type hints in Ovis2 model
thisisiron Jul 4, 2025
8402244
Merge branch 'main' into add-ovis2
thisisiron Jul 4, 2025
2cd3837
Remove commented-out code
thisisiron Jul 4, 2025
1a5f6a9
fix config for tests and remove key mappings
Cyrilvallez Jul 8, 2025
e919722
Update tokenizer configuration to use add_special_tokens method
thisisiron Jul 8, 2025
2de5a94
Merge branch 'main' into add-ovis2
thisisiron Jul 8, 2025
e7e2464
Merge branch 'add-ovis2' of https://github.com/thisisiron/transformer…
thisisiron Jul 8, 2025
d9a8599
skip torchscript
Cyrilvallez Jul 8, 2025
94ba3aa
Fix image placeholder generation in Ovis2Processor
thisisiron Jul 8, 2025
8392223
Merge branch 'add-ovis2' of https://github.com/thisisiron/transformer…
thisisiron Jul 8, 2025
0f19c79
Merge branch 'main' into add-ovis2
thisisiron Jul 9, 2025
d335aaa
Refactor Ovis2 model to rename visual_table to visual_embeddings_table
thisisiron Jul 9, 2025
91e924c
Enhance Ovis2 model by adding vision_feature_select_strategy parameter
thisisiron Jul 9, 2025
3b02fe1
Refactor Ovis2 model weights conversion and architecture
thisisiron Jul 9, 2025
7376160
Refactor Ovis2 model by removing vision_feature_select_strategy param…
thisisiron Jul 9, 2025
683d3e9
Merge branch 'main' into add-ovis2
thisisiron Jul 9, 2025
a8ffbd4
Update Ovis2 model examples
thisisiron Jul 9, 2025
432a718
Refactor Ovis2 model
thisisiron Jul 12, 2025
1d4a1e9
Update Ovis2 model
thisisiron Jul 12, 2025
933cadd
Update Ovis2 model configuration
thisisiron Jul 12, 2025
9ecdd76
Merge branch 'main' into add-ovis2
thisisiron Jul 12, 2025
c024a10
Refactor Ovis2 model test setup
thisisiron Jul 12, 2025
5fb7870
Merge branch 'main' into add-ovis2
thisisiron Jul 16, 2025
3fcdb3a
Merge branch 'main' into add-ovis2
thisisiron Jul 27, 2025
a48468a
Refactor flash attention support
thisisiron Jul 28, 2025
5b02165
Merge branch 'main' into add-ovis2
thisisiron Jul 28, 2025
b5b2eb6
Refactor
thisisiron Jul 28, 2025
5e9c276
Fix typo
thisisiron Jul 28, 2025
0f3163a
Refactor
thisisiron Jul 28, 2025
0c13cfc
Refactor model classes
thisisiron Jul 29, 2025
8d495ee
Update expected output in Ovis2
thisisiron Jul 29, 2025
9d995c3
Refactor docstrings
thisisiron Jul 29, 2025
ccfdb43
Fix
thisisiron Jul 29, 2025
192cc10
Merge branch 'main' into add-ovis2
thisisiron Jul 29, 2025
cfe3a3b
Fix
thisisiron Jul 29, 2025
530aad0
Fix
thisisiron Jul 29, 2025
5d92825
Update input in tests
thisisiron Jul 29, 2025
7bb0e2b
Merge branch 'main' into add-ovis2
thisisiron Jul 29, 2025
c4a83b6
Fix
thisisiron Jul 29, 2025
ac31c2a
Merge branch 'main' into add-ovis2
thisisiron Jul 31, 2025
7b78029
Fix get_decoder method
thisisiron Jul 31, 2025
c230e72
Refactor
thisisiron Jul 31, 2025
3b0a94a
Refactor Ovis2
thisisiron Aug 8, 2025
7cff46b
Merge branch 'main' into add-ovis2
thisisiron Aug 8, 2025
9afdbad
Fix
thisisiron Aug 8, 2025
bd69fb5
Fix
thisisiron Aug 8, 2025
3ed0cb6
Fix test
thisisiron Aug 8, 2025
2b0621c
Add get_placeholder_mask
thisisiron Aug 8, 2025
11802a4
Merge branch 'main' into add-ovis2
thisisiron Aug 8, 2025
38b6f15
Merge branch 'main' into add-ovis2
thisisiron Aug 8, 2025
0e7d6ed
Refactor Ovis2 model tests
thisisiron Aug 14, 2025
7ce5c4e
Fix
thisisiron Aug 14, 2025
0c6571d
Refactor
thisisiron Aug 14, 2025
13010fa
Merge branch 'main' into add-ovis2
thisisiron Aug 14, 2025
2773182
Fix
thisisiron Aug 14, 2025
8642f7d
Fix
thisisiron Aug 14, 2025
62f2023
Fix Ovis2 test
thisisiron Aug 18, 2025
dd47f25
Merge branch 'main' into add-ovis2
thisisiron Aug 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge remote-tracking branch 'upstream/main' into add-ovis2
  • Loading branch information
@thisisiron
thisisiron committed May 5, 2025
commit 11a2a0953ab87ac768847e2ee9433eabcad9b12a
Loading
You are viewing a condensed version of this merge commit. You can view the full changes here.