Jobs for Developers

Technical Duty Officer, Network Operations

BoxRedwood City, CaliforniaFull-time$174k - $218kOct 25, 2023
Apply for this job


Box is the market leader for Cloud Content Management. Our mission is to power how the world works together. Box is partnering with enterprise organizations to accelerate their digital transformation by creating a single platform for secure content management, collaboration and workflow. We have an amazing opportunity to further establish ourselves as leaders in the space, and we need strong advocates to help us achieve that goal. By joining Box, you will have the unique opportunity to help capture a majority of this developing market and define what content management looks like for the digital enterprise. Today, Box powers 100,000+ businesses, including many top Fortune 500 companies who trust our secure collaboration platform to manage the entire content lifecycle.


Box is looking for a dynamic Global Site Reliability Technical Duty Officer to help lead our Global Technical Operations and oversee the continuous health, availability, and reliability of an industry-leading platforms and SaaS offerings.  It is the responsibility of the TDO team to lead 24x7 GTOC teams in preventing, monitoring, identifying, troubleshooting, mitigating, and resolving issues that affect the availability and quality of Box's platforms and services. 

This is an integral shift-based leader and technical escalation point within the GTOC organization, assuming accountability for overall production site health and the performance of core customer facing journeys. This role will help maintain total site awareness, detecting metric and service deviations, providing final level of change approval, and the proactive identification of potential issues; resolving them before they escalate to customer impacting incidents.

We are building a world class Operations Center and need the best talent possible to get us there. That's where you come in!


  • Own and direct live-site Major Incident Management from detection, identification, escalation, mitigation, and recovery.
  • Triage, refine, and verify the Problem Statement, notifies and coordinate the efforts of all appropriate SME resources, and lead cross-functional Incident Bridges to quickly identify and mitigate the problem and restore service.  You'll be evaluated in how well you are able to reduce MTTD to MTTR.
  • Ensure accurate, valid and timely communication to key stakeholders and business entities.
  • Lead daily Incident and Change ticket reviews, coordinate and monitor change windows, and coordinate with Problem Management on critical Issues and action items.
  • Operate across organizational boundaries (Business, Dev, Ops, CS) to protect our customers, their data, and the availability of all Box services, from internal and external security threats, unanticipated volume surges, and significant performance issues.
  • Troubleshoot and identify critical problems in a SOA/API-based, global hybrid cloud, distributed edge architecture on multiple enterprise and public clouds regions.
  • Provide day to day technical expertise and experience to the organization to address issues in globally diverse, high velocity 24x7 environments - from policy and procedural decisions to key architectural and tooling insights to improve Box's Incident, Change, and Problem Management engineering capabilities.
  • Lead daily reviews of planned changes (CAB) in Jira; accountable for reviewing and minimizing change risk, ensuring adequate and appropriate change timing and duration, and complete rollout, validation, and rollback plans that are optimized to prevent site or service impact.
  • Ensure all customer-impacting Incident tickets are completely and correctly documented and augmented with appropriate metrics, timelines, actions taken, and actions still pending.
  • Contributes and reviews Incident postmortems to ensure adequate documentation and appropriate  prioritization of action items related to reducing MTTI, MTTM and MTTR.
  • Participates in Problem Management scrums and Postmortems to identify leading organizational and company-wide technical issues, threats, and trends that block the ability of the organization or teams to perform their roles and provide services optimally and reliably.
  • Lead projects to improve tools and processes related to overall site and service manageability, observability, and resiliency.
  • Coordinate regularly with Infosec, Customer Success, Platform and Dev leaders to continuously access new security and customer on-boarding threats and known issues.
  • Continuously mentor and train Global NOC and system engineers.


  • You have 5+ years of large-scale production/platform operations experience in a large, SaaS provider environments, preferably as a TDO/Major Incident Manager, SRE team leader or Infrastructure (IaaS) or Platform (PaaS) Architecture SME in a Managed Service Provider environment. 
  • Experience in bare metal, Openstack, and Kubernetes architectures supporting a large number of SOA-API-based services.
  • Exposure to Open Source Service-Meshes, Proxies, Caching, Message Buses (Kafka, MQS), NOSQL (Hbase, Hadoop), MYSQL clusters, and Search environments (SOLR, ES).
  • You should be competent in debugging global, distributed Web/API sites based on Linux systems (Ubuntu, RHL, Centos), BGP, iBGP, and IP Anycast networking in multi-vendor virtualized, Edge and hybrid public cloud architectures.
  • You are not expected to be an expert in all areas, but you should be familiar with common terminologies, processes, and architectures in Linux Open Source environments, as well as a thorough understanding of Virtualization, Containers, and Kubernetes.
  • You are confident and comfortable communicating professionally and interacting with individual-contributors through C-level executives from multiple countries, ethnicities, and backgrounds.
  • You have a rock solid command presence and are calm and collected in highly stressful situations, such as a major service outage.
  • You're driven to continuously learn new skills and technologies.
  • Bachelor's degree in Computer Science or Information Systems or equivalent technical field, or similar work experience in a large-scale 24/7 production environment supporting critical, real-time applications.
  • Flexibility to work different shifts and provide weekend coverage depending on need.


  • Solid understanding of ITILv4 Service Lifecycle Management, Service Delivery KPIs, SLIs, SLOs, and Incident, Change, and Problem Management framework, terminology, tools (ServiceNow, Remedy, Jira Service Desk), and processes.
  • Solid knowledge and understanding of security standards and best practices, such as: OWASP, W3C, ISO 27001, SOC1-2, PCI, and SOX. 
  • Ability to troubleshoot secured protocols such as: SSH, SSO, TLS, FTPS, WebDav, HTTPS. 
  • Solid understanding and debugging skills in TCP/IP, BGP, IP Anycast, and distributed internal and external DNS.
  • Two years working experience and knowledge with multi-regional public cloud providers.
  • Experience with observability tools and distributed tracing in large scale environments (Splunk, Datadog, Wavefront, Catchpoint, ThousandEyes, Sensu, SignalFX RUM, Open Telemetry, SNMP).
  • Good understanding and experience with configuration management tools and CI/CD pipelines - Puppet, Ansible, Terraform, Artifactory.
  • Excellent interpersonal and communication skills.


  • Understanding of Agile methods and tools (Jira).
  • Experience with WAF, Bot Managers, and Content Delivery Networks (Cloudflare, Akamai).
  • Experience working in and transitioning into multi-regional hybrid cloud architectures (GCP preferred, AWS).
  • Understanding of Apache Zookeeper and Hadoop. 
  • Experience with large production Scala, Java, Node, PHP environments helpful.  
  • Experience working with various message bus technologies (Kafka, RabbitMQ, MQS).
  • Experience working with relational and non-relational databases  and search engines (Mysql, Postgres, HBase, Elastic Search, SOLR).
  • Experience with caching apps (Squid, Redis, Memcache).
  • Experience with service mesh technologies in a hybrid-cloud environment (Zookeeper, Smart Stack, Istio).

Head-over-heels about this role — but not sure you meet all the requirements? Apply anyway! Studies have shown that women and people of color are less likely to apply to jobs unless they meet every single qualification. At Box, we take a big-picture approach to hiring that fosters authenticity, diversity, and inclusion. If you're passionate about this opportunity, chances are, you shine pretty bright.


We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, disability, and any other protected ground of discrimination under applicable human rights legislation. Box strives to respect the dignity and ‎‎independence of people with disabilities and is committed to giving them the same ‎‎opportunity to succeed as all other employees. Inclusiveness is core to our culture at Box, and we strive to ensure you get the most from your interview experience. Box makes reasonable accommodations for applicants with disabilities. If a reasonable accommodation is needed to participate in the job application or interview process, please complete this form Reasonable accommodations may include scheduling adjustments, document dictation and beyond.

Notice to applicants in Los Angeles: Box, Inc and its related branches will consider for employment, qualified applicants with criminal histories in a manner consistent with the Los Angeles Fair Chair Ordinance.  The Fair Chance Ordinance is provided here

Notice to applicants in San Francisco:  Box, Inc and its related branches will consider for employment, qualified applicants with criminal histories in a manner consistent with the San Francisco Fair Chair Ordinance.  The Fair Chance Ordinance is provided here

For details on how we protect your information when you apply, please see our Personnel Privacy Notice. If you are a California-resident, please read our California Applicant & Candidate Privacy Notice here.

Box is committed to fair and equitable compensation practices. Actual base salary (or OTE if commissionable role) is dependent upon factors such as: knowledge, skill level, experience, and work location. This role is also eligible for equity and benefits. For more information on benefits, check out our healthcare benefits and additional Box Benefits + Perks.
In accordance with OFCCP compliance, here is the Pay Transparency Provision
United States Pay Range
$174,500$218,000 USD

Alternative Jobs