Data Role Compression
I predict that role responsibilities in the data industry will shift from purist to more generalist in the near future.
AI code assistance tools puts downward labor market pressure on “pure” roles like analysts and ETL developers.
Think about a typical data analyst skill set: they can write and debug basic sql queries, use excel, and can use a BI tool like Tableau or Power BI. By all means are these valuable skills! But with the advancements in code assistance tools, there is downward pressure on labor marketability for this group. You can pretty quickly ask AI to spin up queries, validate data, and visualize trends (whether its accurate or not in one shot is up for debate, but you can get to 80% instantly for many analytics tasks at least). With all of this in mind, it would be advantageous for analysts (and similar roles that are mostly downstream from sources) to start thinking about data architectures and infrastructure to stay competitive. Not in a theoretical way, but in a practical one. An analyst does not need to become a full data engineer, but they do need to understand how their data gets there.
The same pattern shows up on the other side. Take a more upstream role like an ETL developer. Historically, the focus was on moving data from point A to point B. Reliability. Scheduling. Basic transformations. All of that still matters, but the expectation is changing. If pipelines can be generated faster and patterns are standardized, then the value is no longer just in movement. It is in making the data usable. Clean models. Clear definitions. Alignment with how the business actually operates.
So both ends of the spectrum start to move inward. The analyst reaches upstream, and the engineer reaches downstream. The boundary between the two becomes less rigid over time. You start to see overlap in responsibilities that used to be clearly separated. From that overlap, a different type of role starts to emerge. Someone who can move through the system end to end. They can pull data from an API, load it into a warehouse, shape it into a usable model, expose it in a BI layer, and answer the question that prompted the work in the first place.
Analysts will become less focused on querying and reporting, and more focused on understanding data models and metric definitions. Their work moves slightly upstream, closer to how data is structured and prepared. Engineering roles shift from just building pipelines to owning how data is shaped and consumed. The expectation expands beyond movement into usability and alignment with business logic. Modeling and ML roles move closer to production, with less separation between experimentation and delivery. Across all of these roles, the boundaries start to blur. Each role covers more of the system, and the handoffs between them become less defined. This happens because AI reduces the effort required for individual steps, which raises the importance of understanding how those steps connect into a complete workflow.
I started on the downstream side. Writing SQL, building dashboards, and working in tools like Microsoft Power BI and Tableau. I felt like I had a strong handle on analytics, and I got curious about what sat upstream. I wanted to understand how the data actually got there. That led me into data modeling, then pipelines, then infrastructure decisions around Snowflake, Airbyte, and containerized jobs. Not all at once. Just following that curiosity and going one layer deeper each time. That path is becoming more common.
The fastest way to move up the stack is to study systems, not just tools. Below are the books that really propelled my career. Designing Data-Intensive Applications builds your foundation. Fundamentals of Data Engineering connects the full lifecycle. The Data Warehouse Toolkit sharpens data modeling. Streaming Systems introduces real-time thinking. Docker Deep Dive bridges into infrastructure, and Kubernetes Up and Running pushes further into orchestration. Then, pair that with hands-on work in a cloud platform like AWS.
Pivoting to systems thinking will transform your career, but also provide more job security, in this changing labor market due to AI.