Big Data Engineer
Our client a world leading Global Systematic Hedge Fund, headquartered in New York, is currently looking to hire talented and dynamic, Big Data Engineer to join a world-leading engineering team based in NYC. Our client is at a tremendously exciting stage with extensive innovative work on-going and an exciting vision to fulfill.
The successful candidate will work on the collecting, storing, processing, and analyzing of huge data sets in a world-leading technical environment. The main objective will be to design the optimal software solutions whilst implementing, maintaining and monitoring them. Some of the roles and responsibilities include:
- Work closely with the financial analysts to architect, design and build real-time big data platform solutions to automate the investment process.
- Process unstructured data into a form suitable for analysis.
- Create and manage large data pipelines.
- Support business decisions with ad hoc analysis as needed.
- To be able to work in teams and collaborate with others to clarify requirements.
- Bsc. In computer science or related discipline.
- Proven track record of architecting distributed big data solutions.
- Strong programming experience (4+ years) in Python or C++.
- Strong SQL expertise (3+ years), including designing data warehouse schemas and tuning performance of very complex SQL queries.
- Expert level experience with Big Data Technologies (e.g. HDFS, Hive, HBase, Spark, Kafka, Yarn, Storm, Splunk).
- Experience processing large amounts of structured and unstructured data at scale (including writing scripts, web scraping, calling APIs, writing SQL queries, etc.).
- Ability to clean and scrub noisy datasets.
- Ability to build custom software tools.
- Experience with AWS/EMR and other web services.
- Experience with statistical analysis, data mining, machine learning, natural language processing, or information retrieval is a plus.
- Experience with Spark and MapReduce is a plus.
- Experience with data visualization (Tableau) is a plus.