It's frustrating how the data discipline is split up into so many different job titles. I regularly have to work with data engineers, data analysts, analytics engineers, data scientists, machine learning scientists, and machine learning engineers. Each view their role really narrowly, so you need to pull in a lot of people to get something done. It's often easier to just do the work yourself.
I get that some specialisation is useful, but I would like to see more T-shaped data folks.
Yeah, as a "T-shaded data folk" (have worked as a data analyst, data scientist, data engineer, and machine learning engineer, so I think I'm close to a full bingo care) in data, I agree!
I think the specialisms are really helpful for hiring - data engineering and data science involve a lot of different things so it's a lot easier to hire someone if you don't require skill in both.
That said, I think we have a culture problem in defining our roles narrowly, I've seen data scientists look down at data-pipeline parts of their job, and ignoring model latency and deployment; data engineers be deeply uncurious around what happens to the data after they put it in a warehouse; etc.
The best people I've worked with, within data and elsewhere, are the ones who are curious about every aspect of the stack, even if they mostly focus on one side.
> Yeah, as a "T-shaded data folk" (have worked as a data analyst, data scientist, data engineer, and machine learning engineer, so I think I'm close to a full bingo care) in data, I agree!
Me too. There were no data engineers when I was a baby data scientist (15 years ago now).
I think that this was a terrible idea (the splitting out of roles). I remember once being asked in an interview if I was able to pull my own SQL or would I need an analyst. That kind of split always struck me as absurd, given that the only way to build a decent model is to know the data very well first..
> That said, I think we have a culture problem in defining our roles narrowly, I've seen data scientists look down at data-pipeline parts of their job, and ignoring model latency and deployment; data engineers be deeply uncurious around what happens to the data after they put it in a warehouse; etc.
I honestly feel like this is driven by the titles. I will note that most of the unsuccessful/bad data scientists I've met were the ones who only cared about the models. As I always said to people, I can run sklearn in a loop over model families if that's all you bring.
> The best people I've worked with, within data and elsewhere, are the ones who are curious about every aspect of the stack, even if they mostly focus on one side.
This is probably just a good signal in tech in general, some curiosity is really required to be successful in this industry.
It used to be the case that a "data engineer" was just a software engineer that was good at data-related things (maybe they were good at Kafka, airflow, DBT, knew why you'd pick postgres for one job and Clickhouse for another, etc).
Then the BI people (who had never written a line of code except maybe some SQL) got jealous, started calling themselves "data engineers", broke everything ("what do you mean 'we need tests???'"), and ruined the title.
I have a different take on this. I have a friend who used to work at a bank as an analyst, and he had a hardcore finance/math background, and was a self-taught R/Python/SQL coder before this whole 'big data' thing became popular a decade ago.
Back then his bank started building a data science/engineering dept, and he jumped ship, and essentially did the same work for 2x the salary. Let me tell you, understanding Black-Scholes and what the data actually means and how to source it, as well as what finance people actually are looking for is a skill that's 1000x as difficult to get as learning R/Python/Pandas etc.
I think there are big risks with just labelling it as "wild west" and let users do whatever they want there.
It's always been possible to massage your data to fit whatwver case you want to make. Previously you had to go through a data analyst and convince them to let go of their principles. Now you can just let an agent figure it out.
So how do you make sure insights are actually shareable (and trustable) with other parts of the company?
I think just exposing the metrics layer is probably the answer here.
It's kind of funny that the industry has tried to democratize data for so long, but now we scramble to restrict access to protect users from making incorrect decisions.
everything old is new again. it used to be the venerable ETL guy who would be the pivotal cog in the machine learning gear. someone who knows where the data is and what it means is always important.
The best quote I've seen about AI and work: "If your job can be done from a laptop, you should be seriously considering what you'll be doing in 5 years".
Depending on the field, five years is generous. 6-12 months (if not already gone) for many things.
I made my 20 year career as a person who could do stuff with computers, software engineer etc. Every time I hear someone say, "my non tech friend who vibe coded an app got a bug he couldn't fix" etc, my response is they are using the wrong model and give it a few months. And I say this with great pain, I am in no way an AI booster, just a realist.
I am wondering whether us folks in the IT space are missing things about how the real world out there functions with those predictions.
As long as we do not hand over our collective decision making power to the machine god, which I don't think is a good idea and am not sure is even possible, the world out there is at this moment still shaped by humans, with other humans, for humans.
I have this feeling that there is more about society and economics than efficiency. I have right now people I know that work in jobs that could have been automated for many years to a very high degree. That is even without AI.
> As long as we do not hand over our collective decision making power to the machine god, which I don't think is a good idea and am not sure is even possible,
Counterpoint: This is not only possible but has effectively already happened. From the outside, the current trajectory is indistinguishable from humanity serving the AI, pulling all possible resources out of literally everything else in order to burn them on AI R&D, training and inference.
It will probably take decades before the world adapts to AI. Even some of the most modern societies still haven’t fully digitilized yet. There will be plenty of time to ride the wave of this adaptation, and still get payed for software work. There’s not enough mechanisms in place required for a fully automated adaptation. Human expertise will remain the missing piece for doing this shift efficiently.
I'm less convinced. I think as engineers/software-folks we tend to look at "something being built" as the end-result (since that's what we work towards).
Big companies though, are more concerned with value and risk. Say some data model provides $5 million in value, hiring a data person to check "everything is kosha", even if in theory someone could vibe code it by prompting an LLM, is a pretty good investment if it derisks the process.
To your point, I wonder if organizations will trend towards hiring 3rd party companies to check work so they have someone to blame and can "focus on innovation". It's already a popular tactic to hire consultants for risky business decisions. Perhaps this is a new prediction: companies get leaner while outsourcing the checks to the same people they fired.
Since the release of Opus 5.5 I have more work than ever and it enables me to move out from the coding I never really enjoyed to a full stack product engineering.
I'm also working on transitioning from a classic SWE to a small independent team doing our own products.
Sure I won't make as much money for doing bullshit changes in some old legacy system but I will actually deliver things people will like using, and I'll be in touch with other people on a daily basis.
I have zero stock or interest in Anthropic IPO, moreover I really hope open weights catch up price-wise to what Claude Max provides (tokens there are heavily subsidized right now), so that I can make a switch and not feed the beast.
I'd like to also use a model on my owm hardware, but alas, I'm relatively young in the industry and my wallet is not fat enough for a local AI stack.
My view, as someone who was flat out "AI is a hype scam" up until around Jan this year. It was hype for a while, and forms of it are still very much hype, but the reality is catching up with the hype.
I.e the reality has so far been ~2 years behind the hype.
I'm not saying I like these companies, or that I think the economics of any of them is sane. Just saying LLM's are capable.
edit. For some context, I only use open weight models these days and rather hope that big closed firms go bust.
I was out for 2 weeks, and the resident meat proxies on my team managed to half-destroy the project by doing everything the way AI wanted to, instead of how the company systems work, introduced like half a dozen cross-connections across barriers that should never be crossed, introduced a parallel artifact repo, permissions system, and different build tooling to what the rest use (including what the current project was built with), and managed to create a lot of drama with external teams who refused to work with these new systems.
AI still needs lots of guidance, and might seem like a genius that does everything on its own, until you give it to people who can't guide it and things begin spiraling out of control.
The same could easily have happened pre LLMs (just slower). Its why things like ArchUnit existed, or style guides, or annoying architects who stopped engineers from using latest JS framework pre LLMs, to define high level requirements for software.
I try to avoid justifying my own existence, but its a reason why I think skilled engineers are way more productive using LLM's than person off the street. Because they can confine them. Though there is no technical reason that ability can't be codified.
Look, I'm a new account, a dev and I'm a human last I checked.
I am not alone in dropping a positive AI message once in a while without having "an account" which is an outdated, useless notion anyway.
I find the greybeards unbearably stuck in anti-AI psychosis. People like myself have been saying LLMs are capable for years and were nothing but stigmatized and ignored. But now ya'll are seeing the light? A little tardy to the party. Again. Always late, but with a big mouth.
Cut a long story short: I don't gaf about Anthropic or any American company for that matter but I also don't gaf what happens to shortsighted big mouth developers that need literal years to see what was obvious on day one. No longer do I need to bow down to these types and it's extraordinarily liberating. In my company the dickhead sysadmins are falling one by one and it's gorgeous. Good luck memorizing manpages and obscure pf settings, because that's what we really need now. Humans memorizing things and calculating because that's what we're really great at. Great long-term strategizing there Newton.
We're now at three fresh accounts and one additional greybearded anti AI strawman.
___
Huh, actually, if I look back on my extensive conversations with LLMs and compare those with human conversations, the whole idea of a greybeard memorizing manpages is something they liked to come up with much more than humans do (if at all).
The idea of a greybeard kinda died out in normal conversation a while ago. Mostly, at least.
> A model doesn’t know that when the sales team talks about revenue they mean commissionable revenue, and the finance team uses something different.
Umm, the model knows the same way humans know: either by looking into slack threads/confluence pages/emails… If your model has access to those things, then it can know. And I foresee that in the near future the model can simply have an incarnation as a Slack member, and when context is not enough, they will simply ask people around (and update docs accordingly)
They asked ChatGPT to invent a new job title for marketing, genius. At a previous company our marketing person created some new term for a segment so we could say we are the only ones doing it. I think there was a term for this grift but I don't remember.
This fake job title sounds like what BI analysts or analytics engineers should be doing as part of their job, not a new job. These jobs are the ones that are usually working on the modeling, answering business questions, and making data accessible and would be creating the context, prompts, and evals to make sure it's working as intended. I think really, analytics engineering and BI should be combined.
I get that some specialisation is useful, but I would like to see more T-shaped data folks.
I think the specialisms are really helpful for hiring - data engineering and data science involve a lot of different things so it's a lot easier to hire someone if you don't require skill in both.
That said, I think we have a culture problem in defining our roles narrowly, I've seen data scientists look down at data-pipeline parts of their job, and ignoring model latency and deployment; data engineers be deeply uncurious around what happens to the data after they put it in a warehouse; etc.
The best people I've worked with, within data and elsewhere, are the ones who are curious about every aspect of the stack, even if they mostly focus on one side.
Me too. There were no data engineers when I was a baby data scientist (15 years ago now).
I think that this was a terrible idea (the splitting out of roles). I remember once being asked in an interview if I was able to pull my own SQL or would I need an analyst. That kind of split always struck me as absurd, given that the only way to build a decent model is to know the data very well first..
> That said, I think we have a culture problem in defining our roles narrowly, I've seen data scientists look down at data-pipeline parts of their job, and ignoring model latency and deployment; data engineers be deeply uncurious around what happens to the data after they put it in a warehouse; etc.
I honestly feel like this is driven by the titles. I will note that most of the unsuccessful/bad data scientists I've met were the ones who only cared about the models. As I always said to people, I can run sklearn in a loop over model families if that's all you bring.
> The best people I've worked with, within data and elsewhere, are the ones who are curious about every aspect of the stack, even if they mostly focus on one side.
This is probably just a good signal in tech in general, some curiosity is really required to be successful in this industry.
Then the BI people (who had never written a line of code except maybe some SQL) got jealous, started calling themselves "data engineers", broke everything ("what do you mean 'we need tests???'"), and ruined the title.
Back then his bank started building a data science/engineering dept, and he jumped ship, and essentially did the same work for 2x the salary. Let me tell you, understanding Black-Scholes and what the data actually means and how to source it, as well as what finance people actually are looking for is a skill that's 1000x as difficult to get as learning R/Python/Pandas etc.
It's always been possible to massage your data to fit whatwver case you want to make. Previously you had to go through a data analyst and convince them to let go of their principles. Now you can just let an agent figure it out.
So how do you make sure insights are actually shareable (and trustable) with other parts of the company?
I think just exposing the metrics layer is probably the answer here.
It's kind of funny that the industry has tried to democratize data for so long, but now we scramble to restrict access to protect users from making incorrect decisions.
I made my 20 year career as a person who could do stuff with computers, software engineer etc. Every time I hear someone say, "my non tech friend who vibe coded an app got a bug he couldn't fix" etc, my response is they are using the wrong model and give it a few months. And I say this with great pain, I am in no way an AI booster, just a realist.
As long as we do not hand over our collective decision making power to the machine god, which I don't think is a good idea and am not sure is even possible, the world out there is at this moment still shaped by humans, with other humans, for humans.
I have this feeling that there is more about society and economics than efficiency. I have right now people I know that work in jobs that could have been automated for many years to a very high degree. That is even without AI.
Counterpoint: This is not only possible but has effectively already happened. From the outside, the current trajectory is indistinguishable from humanity serving the AI, pulling all possible resources out of literally everything else in order to burn them on AI R&D, training and inference.
Big companies though, are more concerned with value and risk. Say some data model provides $5 million in value, hiring a data person to check "everything is kosha", even if in theory someone could vibe code it by prompting an LLM, is a pretty good investment if it derisks the process.
What makes that quote so special that it is worth pointing out as "the best" in 11/2026?
Related: https://news.ycombinator.com/item?id=50438318
Edit: by "semi-auto", I mean auto mode + lots of manual involvement, as opposed to an agent loop or other yolo strategies
I'm also working on transitioning from a classic SWE to a small independent team doing our own products.
Sure I won't make as much money for doing bullshit changes in some old legacy system but I will actually deliver things people will like using, and I'll be in touch with other people on a daily basis.
I'd like to also use a model on my owm hardware, but alas, I'm relatively young in the industry and my wallet is not fat enough for a local AI stack.
I.e the reality has so far been ~2 years behind the hype.
I'm not saying I like these companies, or that I think the economics of any of them is sane. Just saying LLM's are capable.
edit. For some context, I only use open weight models these days and rather hope that big closed firms go bust.
AI still needs lots of guidance, and might seem like a genius that does everything on its own, until you give it to people who can't guide it and things begin spiraling out of control.
I try to avoid justifying my own existence, but its a reason why I think skilled engineers are way more productive using LLM's than person off the street. Because they can confine them. Though there is no technical reason that ability can't be codified.
They weaponize the HN rules here. Which is frustrating, because one has to break the letter to defend the spirit.
Or it could be a kid that watched too many tech YouTubers.
But, fundamentally, that doesn't matter, because functionally it's as described.
I am not alone in dropping a positive AI message once in a while without having "an account" which is an outdated, useless notion anyway.
I find the greybeards unbearably stuck in anti-AI psychosis. People like myself have been saying LLMs are capable for years and were nothing but stigmatized and ignored. But now ya'll are seeing the light? A little tardy to the party. Again. Always late, but with a big mouth.
Cut a long story short: I don't gaf about Anthropic or any American company for that matter but I also don't gaf what happens to shortsighted big mouth developers that need literal years to see what was obvious on day one. No longer do I need to bow down to these types and it's extraordinarily liberating. In my company the dickhead sysadmins are falling one by one and it's gorgeous. Good luck memorizing manpages and obscure pf settings, because that's what we really need now. Humans memorizing things and calculating because that's what we're really great at. Great long-term strategizing there Newton.
___
Huh, actually, if I look back on my extensive conversations with LLMs and compare those with human conversations, the whole idea of a greybeard memorizing manpages is something they liked to come up with much more than humans do (if at all).
The idea of a greybeard kinda died out in normal conversation a while ago. Mostly, at least.
Good job on the abliteration I suppose.
Umm, the model knows the same way humans know: either by looking into slack threads/confluence pages/emails… If your model has access to those things, then it can know. And I foresee that in the near future the model can simply have an incarnation as a Slack member, and when context is not enough, they will simply ask people around (and update docs accordingly)
This fake job title sounds like what BI analysts or analytics engineers should be doing as part of their job, not a new job. These jobs are the ones that are usually working on the modeling, answering business questions, and making data accessible and would be creating the context, prompts, and evals to make sure it's working as intended. I think really, analytics engineering and BI should be combined.