Remote Job
Senior Data Engineer (Scala/Spark)
SemanticBits
-
4 years ago
-
Location : USA
Position SummarySemanticBits is looking for a talented Data Engineer who is eager to apply computer science, software engineering, databases, and distributed/parallel processing frameworks to prepare big data for the use of data analysts and data scientists.You will use Spark to build data processing pipelines that derive information from large sets of government data. You will be a subject matter expert for Spark, the Spark Engine, and the Spark Dataframe API. You will use their knowledge of Spark to teach others, inform design decisions, and debug runtime problems.Tools & TechnologySpark, Hadoop, Scala, Python, and AWS EMRJupyter and ZeppelinAirflow, Jenkins, and AWS Step FunctionsAWS S3, AWS Redshift, and TeradataGSuite, Slack, Jira, Confluence, Git, and GithubResponsibilities
- scala
- devops
- python
- big data
- hadoop
- aws