Software Engineer - CLI Tools Building

Summary

Dice is hiring a Software Engineer to build CLI tools, data pipelines, and automated evaluation systems for its PII Rewrite Service. The role focuses on developing regression test harnesses, provisioning tools, high-volume data ingestion workflows, and resilient integrations with backend and downstream APIs.

Responsibilities

• **Golden Dataset & Auto-Evaluation Pipeline**
• **Dataset Conversion Tooling:** Build the conversion tools required to transform human-rater annotated evaluation results into a structured, machine-interpretable "Golden Dataset" using Protocol Buffers
• **Regression Test Harness:** Build an Auto-Eval Harness that treats this dataset as a regression test suite to automatically evaluate the Rewrite Service and calculate key metrics like PII Recall, PII Precision, and Rewrite Consistency
• **Multi-Format & Multi-Language Scaling:** Ensure the extraction pipeline and evaluation harness support core Workspace assets (Docs, Sheets, Slides, Gmail, PDFs, MS Office, Calendar, Chats) across multiple languages (English, Spanish, Portuguese, Japanese, Korean)
• **OTA Provisioning & Data Ingestion Automation:**
• **CLI Tool Development:** Develop the smb ota_provisioner, a robust command-line tool (in C++, Python, or Go) to orchestrate OTA check, creation, and data upload workflows
• **Automated Provisioning:** Interface with backend systems via Stubby (e.g., TestaccountService) to dynamically create and assign Prod Gaia OTAs and Workspace (Dasher) accounts
• **High-Volume API Ingestion:** Ingest sanitized SMB data from Colossus (CNS) and upload it to the OTAs in parallel by leveraging and extending the internal ota_lifecycle tool
• **Rate Limiting & Resilience:** Implement robust error handling and retry policies with exponential backoff to respect strict write limits on downstream APIs (Gmail, Drive)

Skills

• Strong programming experience in C++, Python, or Go
• Experience building robust, fault-tolerant Command Line Interface (CLI) tools and data processing pipelines
• Familiarity with defining and parsing structured data schemas (e.g., Protocol Buffers, JSON, TextFormat)
• Experience working with RPC frameworks (e.g., Stubby/gRPC) and REST APIs
• Solid understanding of system design principles, particularly around handling API rate limits and implementing exponential backoff mechanisms

Qualifications

Must Haves

• Strong programming experience in C++, Python, or Go
• Experience building robust, fault-tolerant Command Line Interface (CLI) tools and data processing pipelines
• Familiarity with defining and parsing structured data schemas (e.g., Protocol Buffers, JSON, TextFormat)
• Experience working with RPC frameworks (e.g., Stubby/gRPC) and REST APIs
• Solid understanding of system design principles, particularly around handling API rate limits and implementing exponential backoff mechanisms

Benefits

• Remote work arrangement in the USA, with EST or PST time-zone options