add tokenizer + response trimming (1b510bb5) · Commits · AIcrowd / Challenges / Meta Comprehensive RAG Benchmark - KDD Cup 2024 / Meta Comphrehensive RAG Benchmark starter kit · GitLab

Skip to content

GitLab

Explore

Sign in

Primary navigation

Project

Meta Comphrehensive RAG Benchmark starter kit
- Activity
- Members
- Labels
- Issues
- Issue boards
- Milestones
- Wiki
- Environments
- Terraform modules
- Incidents

Snippets Groups Projects

1b510bb5

Commit 1b510bb5 authored 11 months ago by spmohanty

Downloads
- Patches
- Plain Diff

add tokenizer + response trimming

parent 9a7c5b04

No related branches found

No related tags found

No related merge requests found

Changes 8

Expand all Hide whitespace changes

Inline Side-by-side

Showing

local_evaluation.py 16 additions, 2 deletions

local_evaluation.py
models/dummy_model.py 6 additions, 1 deletion

models/dummy_model.py
models/utils.py 16 additions, 0 deletions

models/utils.py
tokenizer/README.md 3 additions, 0 deletions

tokenizer/README.md
tokenizer/special_tokens_map.json 23 additions, 0 deletions

tokenizer/special_tokens_map.json
tokenizer/tokenizer.json 93391 additions, 0 deletions

tokenizer/tokenizer.json
tokenizer/tokenizer.model 3 additions, 0 deletions

tokenizer/tokenizer.model
tokenizer/tokenizer_config.json 33 additions, 0 deletions

tokenizer/tokenizer_config.json

with 93491 additions and 3 deletions

Loading

0% Loading or .

You are about to add 0 people to the discussion. Proceed with caution.

Finish editing this message first!

Please register or sign in to comment