Data Engineer

About This Opportunity

CTI Staffing is partnering with a leading financial services organization to find a Senior Data Engineer for their data aggregation team based in Durham, NC.

This team builds external-facing data solutions that bring together retail, clearing, and custody data to power financial goal planning, investment advice, and projection tools used by partner organizations. It's high-visibility work where data reliability, security, and partner integration quality all matter.

What You'll Do

• Design and build ETL pipelines in Scala/Spark to consolidate and move data across systems
  


• Develop and maintain Java clients consuming REST and SOAP APIs for external partner integration
  


• Schedule, monitor, and debug ETL Spark batch jobs running on AWS EMR
  


• Manage data flow across AWS services including S3, Lambda, SNS, IAM, and KMS
  


• Write to and maintain NoSQL datastores (Cassandra or Yugabyte)
  


• Support Snowflake-based data warehousing workflows
  


• Work with large dataset storage formats such as Parquet and HDF5
  


• Schedule and monitor production jobs using Control-M
  


• Collaborate across teams on data architecture decisions and code reviews
  

Requirements

What You Bring

Must-Have:

• Bachelor's degree in Computer Science, Information Systems, or related field
  


• 8+ years of data engineering experience (10+ preferred), with proven Spark Services development for data movement
  


• Hands-on Scala development for Spark-based ETL pipelines (not Python-based)
  


• Java experience consuming REST and SOAP APIs in data pipeline contexts
  


• Production experience with AWS EMR running Spark batch jobs
  


• Working knowledge of AWS S3, Lambda, IAM, KMS, and SNS
  


• Experience with Cassandra or Yugabyte (or comparable NoSQL datastore)
  


• Experience with Snowflake or a comparable cloud data warehouse
  


• Experience with large dataset storage formats (Parquet, HDF5, or similar)
  

Nice-to-Have:

• Python scripting experience
  


• Background in financial services or other regulated data environments
  


• Experience with enterprise job scheduling tools (Control-M or similar)
  


• Experience building external-facing/partner-facing data integrations
  

Technical Environment:

• Scala, Java, Apache Spark, AWS (EMR, S3, IAM, KMS, Lambda, SNS, RDS), Cassandra/Yugabyte, Snowflake, Control-M, Parquet, HDF5
  

What Success Looks Like:

• Independently owns Spark batch job development and troubleshooting within the first 90 days
  


• Integrates external API consumption cleanly into existing pipelines
  


• Demonstrates fluency across the AWS data stack without ongoing supervision