Salesforce Application Manager

Build workflows for sampling and reviewing high-risk or low-confidence interactions. - Integrate Salesforce with AI platforms, large language models, knowledge systems, data warehouses, and analytics tools.

Join a Global Leader in Workforce Solutions – Net2Source Inc.
Who We Are
Net2Source Inc. isn't just another staffing company, we're a powerhouse of innovation, connecting top talent with the right opportunities. Recognized for 300% growth in the past three years, we operate in 34 countries with a global team of 5,500+. Our mission? To bridge the talent gap with precision—Right Talent. Right Time. Right Place. Right Price.

Description:

Role Name: Salesforce Application Manager
Location: Menlo Park, CA. ONSITE
Duration: 12+ Months

Job Description:

**1. AI Evaluation Framework Development**
- Design and implement a comprehensive evaluation framework for AI-generated employee support responses.
- Establish automated and human-in-the-loop evaluation workflows.
- Develop mechanisms to assess:
- Response accuracy
- Completeness
- Relevance
- Clarity
- Tone
- Groundedness
- Policy compliance
- Employee experience quality
- Build configurable evaluation scorecards and quality thresholds.
- Enable evaluation across multiple use cases, channels, employee groups, and support domains.
- Maintain versioned evaluation datasets, benchmarks, and test scenarios.

**2. Salesforce Application Engineering**
- Design, configure, and develop Salesforce solutions supporting AI quality assurance and evaluation.
- Build custom objects, Apex services, Lightning Web Components, flows, validation rules, and automation.
- Develop reusable services for capturing AI prompts, responses, source references, confidence scores, and evaluation outcomes.
- Extend Salesforce Service Cloud and related employee support capabilities.
- Implement secure, scalable, and maintainable solutions aligned with Salesforce engineering standards.
- Support sandbox, development, testing, staging, and production environments.

**3. Response Quality Measurement**
- Create automated evaluation pipelines for AI-generated and human-assisted responses.
- Compare responses against approved knowledge sources, expected answers, and business policies.
- Develop scoring logic for factual accuracy, relevance, completeness, and actionability.
- Capture evaluator feedback and convert it into structured quality metrics.
- Build workflows for sampling and reviewing high-risk or low-confidence interactions.
- Enable trend analysis by use case, model version, region, policy area, and interaction type.

**4. Hallucination Detection and Grounding Validation**
- Develop mechanisms to identify unsupported, fabricated, or inconsistent AI responses.
- Validate whether responses are grounded in approved enterprise knowledge sources.
- Capture and assess citations, source references, retrieval results, and confidence indicators.
- Flag responses that contain unverifiable claims or contradict enterprise policy.
- Route suspected hallucinations for human review and remediation.
- Partner with AI and data science teams to improve retrieval, prompting, and model behavior.

**5. Regression Testing Infrastructure**
- Build automated regression suites for AI-enabled Salesforce capabilities.
- Maintain benchmark prompts, expected responses, edge cases, and negative test scenarios.
- Compare response quality across model, prompt, knowledge base, workflow, and application releases.
- Detect quality degradation before production deployment.
- Integrate AI evaluation tests into CI/CD and release-management pipelines.
- Establish release gates based on defined quality and compliance thresholds.

**6. Policy and Compliance Validation**
- Translate employee support policies, procedures, and regulatory requirements into executable evaluation rules.
- Build automated checks for prohibited content, sensitive data handling, required disclosures, and escalation requirements.
- Ensure AI responses comply with applicable HR, privacy, security, legal, and corporate policies.
- Maintain audit trails for evaluations, overrides, approvals, and corrective actions.
- Support compliance reviews, audits, and evidence collection.
- Implement access controls and data-retention standards for evaluation data.

**7. Human Evaluation Workflows**
- Design reviewer interfaces and queues for human evaluation within Salesforce.
- Enable quality analysts and subject-matter experts to score, annotate, and classify responses.
- Support blind reviews, consensus scoring, adjudication, and reviewer calibration.
- Create task-routing logic based on risk, business domain, language, and evaluator expertise.
- Capture structured reviewer feedback for model and process improvement.
- Monitor reviewer agreement and evaluation consistency.

**8. Quality Monitoring and Analytics**
- Build dashboards and reports for AI quality and operational performance.
- Track metrics such as:
- Response accuracy rate
- Hallucination rate
- Policy compliance rate
- Regression failure rate
- Human escalation rate
- Reviewer agreement
- Employee satisfaction
- Resolution effectiveness
- Provide drill-down capabilities by model, release, use case, interaction type, and policy category.
- Develop alerts for quality threshold breaches and emerging failure patterns.
- Provide stakeholders with actionable insights and remediation recommendations.

**9. Integration and Data Engineering**
- Integrate Salesforce with AI platforms, large language models, knowledge systems, data warehouses, and analytics tools.
- Develop secure REST, event-driven, batch, and middleware-based integrations.
- Ingest conversation logs, model Clientdata, retrieval context, evaluation scores, and reviewer feedback.
- Ensure data quality, lineage, traceability, and reconciliation.
- Optimize data models and processing pipelines for high-volume evaluation workloads.
- Protect employee and enterprise data through appropriate security and privacy controls.

**10. Testing, Deployment and Production Support**
- Develop unit, integration, regression, performance, and security tests.
- Support user acceptance testing, release readiness, deployment, and hypercare.
- Troubleshoot Salesforce, integration, data, workflow, and evaluation-processing issues.
- Lead root-cause analysis for production incidents and quality failures.
- Implement preventive controls and technical improvements.
- Maintain operational documentation, runbooks, design specifications, and support procedures.

**11. Cross-Functional Collaboration**
- Partner with AI Engineering, Product Management, Data Science, Employee Experience, HR, Compliance, Legal, Security, and Quality teams.
- Translate business quality expectations into technical requirements and evaluation criteria.
- Participate in architecture reviews, sprint planning, backlog refinement, and release governance.
- Communicate risks, quality trends, technical decisions, and remediation plans.
- Provide technical leadership and knowledge transfer to engineering and support teams.
**Required Qualifications**
- Bachelor's degree in Computer Science, Engineering, Information Systems, or a related discipline.
- 5+ years of enterprise application development experience.
- 3+ years of hands-on Salesforce development experience.
- Strong experience with:
- Apex
- Lightning Web Components
- Salesforce Flow
- SOQL
- REST APIs
- Salesforce security and data models
- Experience building applications on Salesforce Service Cloud or Employee Service platforms.
- Experience designing automated testing or quality assurance frameworks.
- Strong knowledge of software engineering, integration, and CI/CD practices.
- Experience working with AI, machine learning, conversational AI, or large language model applications.
- Strong analytical, troubleshooting, and communication skills

**Preferred Qualifications**
- Experience building evaluation infrastructure for generative AI or conversational AI systems.
- Knowledge of AI evaluation concepts such as groundedness, relevance, factuality, hallucination detection, and model regression.
- Experience with Salesforce Einstein, Agentforce, Data Cloud, or comparable AI-enabled Salesforce capabilities.
- Experience integrating Salesforce with enterprise knowledge bases and AI platforms.
- Familiarity with Python, SQL, data pipelines, analytics platforms, and automated evaluation libraries.
- Knowledge of prompt management, retrieval-augmented generation, model observability, or LLM operations.
- Experience supporting HR, emplo"

Why Work With Us?
We believe in more than just jobs—we build careers. At Net2Source, we champion leadership at all levels, celebrate diverse perspectives, and empower you to make an impact. Think work-life balance, professional growth, and a collaborative culture where your ideas matter.
Our Commitment to Inclusion & Equity
Net2Source is an equal opportunity employer, dedicated to fostering a workplace where diverse talents and perspectives are valued. We make all employment decisions based on merit, ensuring a culture of respect, fairness, and opportunity for all, regardless of age, gender, ethnicity, disability, or other protected characteristics.
Awards & Recognition
• America's Most Honored Businesses (Top 10%)
• Fastest-Growing Staffing Firm by Staffing Industry Analysts
• INC 5000 List for Eight Consecutive Years
• Top 100 by Dallas Business Journal
• Spirit of Alliance Award by Agile1
Ready to Level Up Your Career?
Click Apply Now and let's make it happen.

Best regards,
Juan Carlos Matus 
Sr. Customer Sucess Manager