Almost all techies are familiar with Moore's law. Similarly, peple argue that all technological innovation is happening exponentially. Not to mention the amount of data we are generating is staggering! This statistic has been floating around the internet since 2013 - 90% of the world's data was generated in the last two years. If that's not exponential growth I don't know what is.

The Red Queen Hypothesis

But I'm going to be honest y'all. I love tech, but I'm exhausted. I was first introduced to The Red Queen Hypothesis from a Quora answer about tech and it really spoke to me. The name comes from Lewis Carrol's Through the Looking Glass, with the statement "Now, here, you see, it takes all the running you can do, to keep in the same place."

I am learning so much, but no matter how quickly I absorb knowledge and I don't feel like I am ever an expert because the field evolves even faster than I do. I guess that's why computer and data scientists make the big bucks... Speaking of Quora, there was another senior techie lamenting about the scam that is "Full-Stack". To paraphrase, he said that back in his day being "back end" was an entire career and "front end" was its own career. According to him, "Full-Stack" is a scam to make junior levels do more for less. And I sympathize, I do. I would really like a better work life balance as well. But as our hardware has evolved, so too must our software.

So here were some of my assumptions and also how I was wrong... I thought I was going to be a programmer and I wouldn't have to worry about my environment, that was for the IT people. NOPE! Almost always, you end up your own IT person, especially if you're not in a large tech firm. Funny story, one time I needed help from the IT staff of another company. His response was "I gave you admin rights you can do whatever you need to do." Um... never done that before but okay. I can't help but wonder if he was just lazy or didn't know how to do it. Had to teach myself how to open and close ports within a day to get setup going. Often you gotta manage, and troubleshoot, your own environment. Gotta learn networking or else your product will never make it live online. This probably wasn't even a consideration for programmers in the 90's (like Quora responder) because internet connectivity was not ubiquitous like it is now. I don't even know what it's like to not deploy my own wares. I was incredibly disadvantaged on the job market when I graduated because I didn't know Heroku. I believe it's replacement now is Docker.

The Current State of Tech

I am going to mega dumb this down because that's just the limits of my understanding. Please feel free to correct me in the comments section. Technology careers are split between software (Computer Science) and Hardware (Information Technology). But today, more and more programmers need IT skills (networking, deployment) and more and more IT personnel need programming skills (scripting, PHP). The reason being that software products and the infrastructure required to support them are becoming increasingly complex due to rising demand and performance considerations. This article sums it up really nicely, How It Feels to Learn Javascript in 2016. Yes there have been updates to the article by others since then but this spot on in terms of how it feels. And if you click and get hit by Imposter Syndrome flavored with a touch of despair, that's exactly how I felt too. This is so different from the days when I thought vanilla CSS was magic. But it's okay, no one knows everything and what matters is that you try and keep learning.

In the scope of data science, my computer science field of depth, you may end up needing to know concepts like GPU programming (because machine learning trains faster on the GPU than the CPU). But maybe not because cloud services like AWS and Microsoft Azure can take care of that for you. At a price. The cloud, which btw, is it's own sphere of computer science depth that I am incredibly weak in. Funny story when IBM published an article about "serverless computing" I was super stoked until I found out they were talking about the cloud. Which is literally just someone else's computer somewhere.

Which once again circles back to Cybersecurity - how much can you trust your stuff to be on someone else's computer? But Cybersecurity is once again, a seperate sphere of computer science depth. Except when it's not! Because TAMU has made the news recently for pairing up students with AI (data science, yay!) to fight cybersecurity threats.

Which brings me to HPC, high performance computing, which many data scientists use to handle their computations. But good luck connecting to the HPC and not breaking anything if you have no knowledge of networking. With the advent of big data if you can't run in parallel your programs will run for eternity (not literally). Luckily it's easier than you'd think! Thanks innovation, for making really difficult things much easier than the ordeal I had to go through in undergraduate. (Check out PLINQ if you're into C# and the future package if R is more your thing. Data.Tables is my lifeblood right after the tidyverse). Not to mention that GPU programming is becoming a thing. So is translating between languages.

I guess in the end my conclusion is it's impossible to know it all, but it doesn't hurt to try. It can feel overwhelming but we're all human. It is hard to be in this field if you don't genuinely love it, but since I'm assuming you do just enjoy the journey no matter how tough it can be.

Do you have a minute to talk about Data?

"Where have all the database administrators gone?" I thought to myself. I strongly suspect they've been rebranded to data engineers. A recent article about the top 10 highest paying tech jobs in 2019 are heavily skewed toward data. Among the list are Big Data Engineer, Data Scientist, Data Architect, Data Manager, and Data Security Analyst.

Data careers are pretty poorly defined so I'm not going to go too deeply into which one to choose or how to get there. But the best career advice that I was given was to just look at what jobs are available in your city of choosing and what technologies they're asking for.

I hope you find the optimal data science sphere of depth for your interests. But if not they're all interconnected anyway so it doesn't hurt to collect more. Happy Holidays and good luck on finals.

---
Jennifer Cai
Jennifer is a Masters student in the College of Science's Analytics program.