How I became a data engineer (and how maybe you can too)

Ryan Howe
5 min readMar 17, 2023

I haven’t written on here for a bit and this will be a significant departure from my previous content where I typically just wrote about math and data analytics to something more about myself. Last week I had a software engineer reach out to ask about how I became a data engineer and so we video chatted and I figured I would recap some of that. I’ve had people reach out about similar things in the past but never video chatted with anyone.

Tell me a little about your work experience?

I started doing data related work in 2020 (February) right before the pandemic as a data analyst at a small music company (TuneCore). I taught myself Python, R and SQL a little before this using Coursera and Udemy. When I started I had very little knowledge of data analytics or music but was able to quickly learn a lot there. For most of the time while I was there I had no knowledge of data engineering but slowly learnt that a lot of the work we did relied upon key data engineering ideas to process royalties and make reports.

Eventually a chunk of the data team ended up moving on and a recruiter reached out about a contract position at Facebook/Meta for data engineering. I passed the interviews and joined a subset of ads called advertiser automation. I quickly ramped up and worked on a number of alphas for Automated App Ads and Automated Shopping Ads then later supported various revenue and ecommerce related workstreams (as well as several war rooms) building pipelines, dashboards and alerting for various teams. I got to work on several efficiency and Better Engineering related projects I was interested in as well that I’m proud of. In particular I helped drive a chunk of an efficiency related workstream that saved over $100 million in costs for storage and compute across multiple orgs.

I learnt a great deal about various data related tools and technologies that are open source like Hive, Presto and Spark as well as internal tools like Unidash, Scuba, Dataswarm and many other tools. I enjoyed my time there but was eventually let go in February abruptly and now work at a healthcare related start up doing data engineering.

Data analysts aren’t generally that technical, how did your data analyst experience help?

I think this is generally true however many analysts know SQL and some scripting language and…