Once I was inspired to switch to data engineering, I noticed something interesting: there appeared to be a lot of data engineers like me.
My name is Sam Hoffman, and I'm a data engineer at Particle Health. I work on transforming complex health data formats into other health data formats that are slightly more usable. I love my job, but I had never even heard of it when I first started thinking about what my career might be. The idea of studying computer science seemed way too intimidating for me; I thought it was reserved for boys who were good at math. Luckily, I had a bunch of other interests, and I decided to follow those instead.
Thanks to a State Department program, I spent the year after high school learning Turkish in Samsun, Turkey. At the time, I thought I might try to join the State Department, or at least pursue a career where Turkish was a requirement, and so I majored in Turkish and statistics in college.
When I had to start making decisions about what I wanted to be when I grew up, I wasn’t so sure anymore about the paths available to people who studied Turkish. Instead, I chose to pursue a career as a data scientist, which the Harvard Business Review (potentially prematurely) declared the sexiest job of the 21st century back in 2012. Data science allowed me to creatively explore data sets, work with data way bigger than my college statistics classes had ever dreamed of (30-item data sets were considered to be a large sample size), and communicate my insights with others.
But as time went on, I started to hit the limits of data science. They always had something to do with the data gathering, transformation, storage, and loading processes - in other words, the work of a data engineer. My data science colleagues doing the really sexy work of model building had first spent months and sometimes years carefully curating their data, and more importantly, the systems that collected and shaped their data.
Inspired, I dove headfirst into the work of a data engineer: building up my Python skills, learning Airflow and Spark, and creating data storage models that enabled the analyses I wanted to do. After a couple of months of more engineering-y work, something unexpected happened: I started to be more interested in the engineering than the data science I had meant to enable through my engineering.
I loved writing and debugging code, discussing it with my coworkers in pull requests, and building larger systems piece by piece. I especially loved the fact that there was always something new to learn. I decided to switch to explicitly being a data engineer, rather than a data scientist doing a lot of engineering work.
Once I made the switch, I noticed something interesting: there appeared to be a lot of data engineers like me.
In my experience, data engineering seems to be full of other people who studied something other than computer science, started working on data problems, built up their data engineering skills to solve them, then ended up liking data engineering more than the work they were originally trying to enable through data engineering.
I've had (and looked up to) a lot of coworkers and friends who fit this profile. I think a lot of women follow this path, especially women like me who thought coding wasn’t for them. In my case, I was too intimidated because math didn’t feel easy to me and I thought computer science would be even harder.
Particle Health is my second job where my job title is officially data engineer. My Turkish teachers (and 18-year-old self) would probably be surprised to see where I've ended up, but I'm really happy to be here. I'm learning a lot of new things (my favorite part of the job) and looking forward to building a super cool data processing system that we're in the process of planning right now. While my background isn't traditional for an engineer, I still think my studies in Turkish helped with my engineering work. Nothing teaches the importance of clear communication like trying to say something in a foreign language where you've only learned 20 words. And in my admittedly biased opinion, engineering teams benefit from having people with all kinds of backgrounds.
I do sometimes wish I had known how fun coding could be, or that I hadn’t put off my few required computer science classes until my last year of school out of fear they’d be too hard. I wish I had felt like software engineering was a path available to me, and to women in general. But I’m grateful that I and so many of my smarter and more talented colleagues fell into data engineering, and to data engineering itself for being such an inspiring and fun discipline. I’m also grateful to Particle’s engineering team, which has been feeding my hunger for new knowledge and more engineering skills by adopting me as a data engineer who moonlights in the backend codebase.
Speaking of the engineering team, we're growing! I would love to work with another Turkish-speaking engineer, or someone with backgrounds and experiences I can't even imagine. We have tons of open roles that you can check out, right here.
Electronic health records are great, but it’s the way that you use them that matters. Meaningful Use, a concept from 2009’s HITECH Act, incentivized healthcare organizations to actually utilize the digital health tools which they purchased.
This October, the Anti-Information Blocking Rule will spur new innovation - if the government enforces your right to access your own medical records.