Content Evaluator

Lancesoft

Remote, CA(remote)

Apply

JOB DETAILS

SALARY

SKILLS

Document Imaging, Elasticsearch, English Language, Python Programming/Scripting Language, Scripting (Scripting Languages)

LOCATION

Remote, CA

POSTED

5 days ago

Pay rate: PR $40 - $45/hr

Title: Content Evaluator
Duration: Remote
Location: 5 - 6 month contract

Job Description: As a relevance evaluator for Dash you will be responsible for evaluating how relevance a given document (or image, video) is for a particular user query.
You will have to rate the document relevance 1-5 (from not relevant at all to exactly what the user was looking for).
Rating relevance requires first trying to understand the user intent, for example, where they researching a topic, or they were looking for their own document, and forming an opinion what would be a good document (or image, etc.) for this particular query for this particular user.
Other similar tasks involve evaluating quality of Dash assistant answers, including various aspects of the answer: is it correct (the information answers the user intent correctly), is it descriptive, or is the system apologizing for not knowing the answer. Understanding correctness also requires some research on the terminology used.
Responsibilities:
Responsibility includes reviewing the query and document pairs, including any context provided by the system (such as when the query was issued and who did it), researching the problem trying to understand the user intent by using the product (Dash assistant and search, including formulating queries for elastic search index in KQL), and then rating the document or other required artifact (e.G. Assistant answer) according to the provided relevance rubric.

Requirements:
Understanding of English
Ability to formulate search or chatgpt-like queries

Preferred Qualifications:
No engineering experience is required, but being able to script certain tasks in Python can be beneficial.
Previous experience with relevance labeling or rating would be preferable.
Comments for Suppliers:

About the Company

Lancesoft

We are a $125 Million, NMSDC-certified Minority & Woman owned Workforce Solutions Company headquartered in the DC metro area with presence across US with global presence - Canada, Mexico, India, UK, Malaysia, Indonasia, Hongkong, Singapore, UAE. We are specialized in providing Workforce Solutions, SOW project delivery, Engineering Solutions, Creative Services. We currently support 100+ Fortune companies globally and across multiple industry segments. We are currently supporting several massive programs across industry segment nationally/globally (Intel, Ally, AMD, QUALCOMM, Morgan Stanley, Kraft/ Mondelez, MNP, Amdocs, Dell, SanDisk, Medtronic, Becton Dickinson, GE, Lockheed Martin, UTC, L-3 Communications, Caterpillar, BMW, Mercedes Benz, National Grid, Dominion, Energy Future Holdings, PSEG, 3M, Fidelity, Aetna, Humana, Johnson & Johnson, Pfizer, Merck etc).

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender, identity, national origin, disability, or protected veteran status.

COMPANY SIZE

2,000 to 2,499 employees

INDUSTRY

Staffing/Employment Agencies

FOUNDED

2000

WEBSITE

http://www.lancesoft.com/