Data Engineer
Innovative Defense Technologies

Arlington, Virginia


Background information:

Innovative Defense Technologies (IDT), provider of automated software testing, data analysis, and cybersecurity solutions for complex, mission-critical systems in the US Department of Defense (DOD), is seeking a Data Engineer to be based in our Mt. Laurel office.


The Data Engineer position offers the opportunity to join our growing team of data scientists and analytics experts. The hire will be responsible for designing and implementing various data pipeline architectures across several projects, as well as optimizing data flow and collection for cross functional teams. The candidate must be experienced in the state-of-the-art data management technologies to support differing data requirements and objectives. Individuals must be self-directed and comfortable supporting the data needs of multiple teams, systems and products.

All applicants must be able to obtain/maintain an active U.S. Security Clearance.

Responsibilities Include:

  • Design, implement, deploy and maintain optimal data pipeline and data management architectures.
  • Assemble large, complex data sets that meet functional / non-functional project requirements
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, streaming and 'big data' technologies
  • Implement data pipelines to ingest data to the platform, standardize and transform the data
  • Support the development of analytics tools that utilize the data pipeline to provide actionable insights into the data and optimization of objectives.
  • Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
  • Work with data and analytics experts to strive for greater functionality in our data systems.
  • Design and architect solutions with Big Data technologies (e.g Hadoop, Hive, Spark, Kafka)
  • Design and implement systems that run at scale leveraging containerized deployments
  • Design, build, and scale data pipelines across a variety of source systems and streams (internal, third-party, as well as cloud-based), distributed/elastic environments, and downstream applications and/or self-service solutions
  • Ability to travel up to 10%
Minimum Required Qualifications:
  • Master's Degree in Computer Science, Computer Engineering, Informatics, Information Systems or another quantitative field
  • 5+ years of experience in a Data Engineer role
Required skills:
  • Experience with big data tools: Hadoop, Spark, etc.
  • Experience with relational SQL and NoSQL databases, including Postgres
  • Experience with AWS cloud or remote services: EC2, EMR, RDS, Redshift
  • Experience with stream-processing systems: Kafka, Storm, Spark-Streaming, etc.
  • Experience with object-oriented/object function scripting languages: Python, Julia, Java, C++, Scala, etc.
  • Experience with data encryption/security features applied to data-in-transit
  • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases
  • Experience building and optimizing 'big data' data pipelines, architectures and data sets
  • Experience performing root cause analysis on internal and external data and processes to answer specific questions and identify opportunities for improvement
  • Experience with development, management, and manipulation of large, complex datasets
  • Experience with database & ETL technologies
  • Demonstrated knowledge of data management competencies and implementation
  • Understands the Big Data related problems and requirements to identify the correct technical approach. Core understanding of Big Data principles and architectural patterns
  • Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores
  • Experience supporting and working with cross-functional teams in a dynamic environment
Preferred skills:
  • Experience with containerized deployment technologies (Kubernetes, Openshift, etc.)
  • Experience with instantiating and configuring Virtual Machines (VMware, VirtualBox, etc.)
  • Experience with Machine Learning algorithms and applications interfacing with data management solutions
  • Solid analytical abilities, coupled with a strong sense of ownership, urgency, and drive
  • Attention to detail
  • Initiative, creativity, reliability, teamwork
  • Ability to adapt to and thrive in a fast-paced environment
EEO Statement:

IDT is an Equal Opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, creed, sex, sexual orientation, gender identity, national origin, disability, protected Veteran status, or any other basis protected by federal, state, or local law.

Job Alerts

Provide an email, zip code for jobs, and/or job category to subscribe to job alerts. Learn more now.

*By subscribing, you agree to our Terms and Privacy Policy.

More IT jobs

Optiv Security
Atlanta, Georgia
Posted 8 minutes ago
Optiv Security
Atlanta, Georgia
Posted 8 minutes ago
Optiv Security
Columbia, Maryland
Posted 8 minutes ago
View recent jobs »

New post from our employment blog

New Post

Your Budget Friendly Guide to Working From Home for the First Time

Does suddenly working from home have you discombobulated? Even if you’ll only be working from home for the next few weeks or months, it’s impo...

Read blog post
Share this inclusion job with the community

Click a community link below, and then social share the Data Engineer job.

Disability inclusion jobs logo
Asian inclusion jobs logo
Black inclusion jobs logo
Diversity inclusion jobs logo
LGBTQ inclusion jobs logo
Seniors inclusion jobs logo
Women inclusion jobs logo
Hispanic inclusion jobs logo