feat: project freelancer ranking#40
Conversation
Mattdl
left a comment
Mattdl
left a comment
There was a problem hiding this comment.
Really great PR Warren, those are some really impactful contributions to WorkRB!
I left some comments in the review. One additional change to add, is adding your 2 datasets to the main README in the 'Tasks' overview. This way your datasets will be easily discoverable.
| JA = "ja" | ||
| KO = "ko" | ||
| ZH = "zh" | ||
| CROSS = "cross_lingual" |
There was a problem hiding this comment.
We're currently implementing a refactor that will support the cross-lingual case in #34. But we can keep this label for now and remove it later to make clear that this dataset supports the cross-lingual case.
| PROJECT_BRIEF_STRING = "brief_string" | ||
| """ | ||
| A few sentence describing a project | ||
| """ | ||
|
|
||
| SEARCH_QUERY_STRING = "search_query_string" | ||
| """ | ||
| A few input word to search for a profile | ||
| """ | ||
|
|
||
| PROFILE_STRING = "profile_string" | ||
| """ | ||
| A few sentences describing an applicant profile | ||
| """ |
There was a problem hiding this comment.
I like the 3 new modalities being introduced!
My suggestion would be to make them much more descriptive. When someone in the codebase comes upon these labels, it should be clear what they refer to and what they are used for. Additional context that would clarify this are: (i) in which context is it used? (ii) what is a 'project, query, and profile' in this setting. (iii) how do these label types typically relate to eachother, and (iv) an example for each type.
There was a problem hiding this comment.
I’ve expanded the comments to provide more context while aiming to keep them readable. Please let me know if you feel anything is still unclear.
src/workrb/types.py
Outdated
| A few input word to search for a profile | ||
| """ | ||
|
|
||
| PROFILE_STRING = "profile_string" |
There was a problem hiding this comment.
Renaming to APPLICANT_PROFILE_STRING would avoid ambiguity with possible future setups that contain profiles.
There was a problem hiding this comment.
I renamed it to CANDIDATE_PROFILE_STRING to reduce ambiguity.
I’m also fine with APPLICANT_PROFILE_STRING if you think that aligns better with future extensions.
2 participants
Description
Created a project brief and a search query candidate ranking tasks.
Those tasks evaluate a model's ability to rank freelancer candidates by semantic similarity (skill matching fit) to search query or project. Those tasks rely on a dataset provided by Malt containing synthetic candidate profiles, project briefs, and scored skill fit.
Checklist