Pr/multilingual#121
Draft
raghavm243512 wants to merge 5 commits into
Draft
Conversation
raghavm243512
force-pushed
the
pr/multilingual
branch
from
bd0e0d9 to
81923ab
Compare
raghavm243512
force-pushed
the
pr/multilingual
branch
from
68fb05a to
f5a5b52
Compare
raghavm243512
force-pushed
the
pr/multilingual
branch
from
606dc7d to
ada9699
Compare
Collaborator
katstankiewicz
left a comment
katstankiewicz
left a comment
There was a problem hiding this comment.
can you also add ensure_ascii=False to AuditLog save()
| def get_initial_message(language: str) -> str: | ||
| """Return the assistant's opening line for ``language``. | ||
|
|
||
| Falls back to English. Raises if even English is missing (data quality). |
Collaborator
There was a problem hiding this comment.
Suggested change
| Falls back to English. Raises if even English is missing (data quality). | |
| Falls back to English. Raises if even English is missing. |
| user_message = resolve_user_goal( | ||
| record.user_goal, | ||
| record.culture_overrides, | ||
| os.getenv("EVA_LANGUAGE", "en"), |
Collaborator
There was a problem hiding this comment.
Suggested change
| os.getenv("EVA_LANGUAGE", "en"), | |
| language, |
| goal = resolve_user_goal( | ||
| record.user_goal, | ||
| record.culture_overrides, | ||
| os.getenv("EVA_LANGUAGE", "en"), |
Collaborator
There was a problem hiding this comment.
Suggested change
| os.getenv("EVA_LANGUAGE", "en"), | |
| language, |
and add language to build_user_sim_prompt
| ), | ||
| audit_log=audit_log, | ||
| api_key=params["api_key"], | ||
| base_url=params.get("url", ""), |
Collaborator
There was a problem hiding this comment.
Suggested change
| base_url=params.get("url", ""), |
we don't need a url for openai services
| from eva.utils.culture import FIRST_NAME_PLACEHOLDER, LAST_NAME_PLACEHOLDER | ||
| from eva.utils.json_utils import extract_and_load_json | ||
| from eva.utils.llm_client import LLMClient | ||
| from eva.utils.logging import get_logger |
Collaborator
There was a problem hiding this comment.
Suggested change
| from eva.utils.logging import get_logger | |
| from eva.utils.logging import get_logger, setup_logging |
| from eva.utils.logging import get_logger | ||
| from eva.utils.router import init | ||
|
|
||
| logger = get_logger(__name__) |
Collaborator
There was a problem hiding this comment.
adds logging (previously was only printing warning level or above)
Suggested change
| logger = get_logger(__name__) | |
| setup_logging() | |
| logger = get_logger(__name__) |
|
|
||
| # 3. Write back atomically. | ||
| if dry_run: | ||
| logger.info(f"[dry-run] would update {dataset_path} ({len(records)} records)") |
Collaborator
There was a problem hiding this comment.
Suggested change
| logger.info(f"[dry-run] would update {dataset_path} ({len(records)} records)") | |
| logger.info(f"[dry-run] would update {dataset_path} ({len(target_ids)} records)") |
| def _load_names_file(path: Path) -> dict[str, list[str]]: | ||
| data = json.loads(path.read_text(encoding="utf-8")) | ||
| for key in ("male_first", "female_first", "last"): | ||
| if not isinstance(data.get(key), list) or not data[key]: |
Collaborator
There was a problem hiding this comment.
should this check that there are enough names as well? ie BUCKET_SIZE?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
initial multilingual version
Easily extendable to many language using the add_culture_data script. This will do translation, gender consistent naming, suggest names, extend data, etc. So if anyone wants to run a language not committed in EVA data, it is trivially easy to do so
Readme section showing basic of adding a language.
Tested end to end once, but I don't have 11lab login and ran out of credits on free one (needed a custom agent for foreign lang)
Still TODO:
Currencies
Actually committing the translations (didn't want to burn credits until finalized)
Analysis
Testing a large variety of models to ensure they actually get the language code they expected (es-MX vs es, for example)