(Sept 2024: UPDATE at the end)
Wow! and Holy moley… (if you’re in that camp)
Talk about disappointment, the job market is DEFINITELY tough right now for AWS Data Engineers overall. Tough AND Competitive….
So far, this Data Engineer (DE) journey has cost me a VERY significant amount of money for the past year (no salary and living on savings in our home). All in the belief in my self-confidence (which I very much still have) of getting into Data Engineering, see below.
The oddest part for me though, ~85% of the emails/calls I receive, they are for Senior or Lead DE and/or Data Scientist roles. It is very tough (and disappointing) for me when I have to tell the recruiters — I am trying to break in at the mid-level DE role because I know I do not yet have the Senior level experience yet.
- But give me 6–8 months and it will be a different story…
Do not get me wrong, if the companies find someone more qualified and at a lower salary, good for them. Just not good for me — that is the way it goes. All one can do is — keep on pushing, someone with vision in some company will see the drive and sacrifice (more below) you have endured / incurred / undertaken….
Yet, after gaining two of the ‘most’ desired DE certifications, I had believed I would be a shoo-in for that role, even with my ‘not’ yet having 2, 3–10 years ‘hands-on DE work experience’ most of the want ads indicate. When many of the folks working in DE do ‘not’ yet hold a single DE cert.
Certs obtained:
- AWS Data Engineering (Jun 2024)
- GCP (Google Cloud Platform) Pro Data Engineer (Sep 2023)
Or, it could be the lack of vision in the various company Personnel Departments (many people like the HR cattle term — human resources — I like to refer to people as people and not just resources). Or, even from the viewpoint of the lower-level team members who do the team interviews.
My wish is in actually talking with more senior leaders, those who have a much better strategic and / or tactical aspect. Whenever I’ve done that, I by far have more success.
Okay, the reason why I say this, is that it could be that some of these companies are just ‘NOT’ seeing my sacrifice and drive — which is as clear as day.
I took off a year to pursue the two DE certifications, when I had originally only meant to take off five (5) months, due to my high optimistic goal of success.
This time off from work stemmed from commuting for 2–3 hours a day in this region’s traffic — unavoidable. I had already done that kind of commuting for 15 years in multiple jobs in this area. And taking the metro-rail system as an alternative — nah…. It is equivalent or worse. At the end of the day, I was too burnt out to study every night and many weekends.
The best way for me to get the certifications — stop working to get it done faster. If I had not stopped working, I would still be working on earning the second certification. So, I stopped working and commuting.
I had believed after gaining the GCP certification and then spending three (3) months on learning SQL / Python, it would be a cake walk to get into the GCP world.
One seriously huge surprise I had not accounted for, was when I talked to a couple of Python experts, they told me that if I studied and worked for 6–12 months, I’d at best be at the Proficient level. Some folks would get lucky and become experts, depending on the work they did.
And then guess what, Covid slammed me down for over 3 weeks, bad timing. Lady Luck is having fun with me, lemme tell ya…..
Then, guess what else, the job market hit — big tech layoffs really affected my transitioning over to the GCP world. Note, I was trying to get a workplace close to home due to the pain of commuting here. And I wanted to be “IN” the work place and not fully remote — I prefer hybrid at best. But there just are not enough GCP Data Engineer jobs near me.
Because of the tech market slump, which we have no control over nor foresight in seeing ups and downs — I had to get back to work, so I jumped back to the AWS world where I had the most experience, to obtain the AWS DE cert.
My next surprise was when I took the AWS DE test the first time — I was shocked I did not pass. It was supposed to be at the associate level. So, I hunkered down again and passed it. The AWS cert is a very, VERY tough little nugget, especially if you do not have the 2–3 years of ‘hands-on DE work experience’ that AWS recommends — which I do not have.
But I passed. And have since started working on my AWS ML Specialist certification while looking for an AWS DE role.
At this point, it is very unclear to me as to what companies are looking at.
Or, it could be the lack of vision for many in Personnel or lower-level teams — my background in total is not being considered.
Nine years of working in the AWS Cloud Architect / Engineer space and holding these additional certifications:
- AWS Cloud Architect Pro (renewed Sep 2022, renewed, not a first-time cert)
- AWS Data Analytics Specialist (Dec 2021)
- ** While gaining the Data Analytics cert in 2021, I also stood up AWS Redshift for the Data Scientists on the team — had to teach myself some SQL / Python and pgAdmin to help the Data Scientist team (definitely thought this would aid in my Data Engineer job search)
- ** currently working on AWS ML (machine learning) cert
Also in the past year, picked up more knowledge in:
- pgAdmin
- VScode
- Jupyter Notebook
- Pandas
- Numpy
- Thonny
- and a little bit of LaTeX…
Double graduate degrees:
- MBA in Technology, focus on Strategy & Innovation
- MSc Cybersecurity
Multiple years of working in the:
- Intelligence space, where strategy/tactics are paramount.
- IT space — many years of IT pre-cloud: network / data communications, routers, firewalls, NOC & SOC — and being in leadership roles in many of them
- Cybersecurity space and holding these certifications:
- — — (ISC)2 CISSP (renewed multiple times, not a first-time cert)
- — — (ISC)2 ISSEP (a concentration, also renewed multiple times, not a first-time cert)
Communication and diplomacy skills — while not being the best writer or the best orator, I have engaged with many very senior leaders over the years:
- Ambassadors: In the past, I did multiple overseas visits to do one-on-one cybersecurity talks with several Ambassadors and their Deputy Chief of Mission
- Regional INS Border Patrol Commanders, the number one person for each sector, for cybersecurity
- Engaging in doing briefings, intelligence, with multiple executive leaders (civilian and military), while also letting various team members on my staff take turns to do briefings (so they can get experience) — one briefing included the head and deputy of an agency with only 12 of us in that, ummm, quiet meeting
- Engaged with all division level leaders at an agency to clean up their AWS Cloud assets and to stop wasting money every month (or course, it was not quite worded that way…) — saved that agency a LARGE amount of revenue
- etc., etc., etc…..
And let’s talk about cognitive diversity skills — my brain being one of damn good plasticity, the brains’ capability to learn more and do more… Always looking for better, creative, productive, successful outcomes.
The more diverse subjects/topics you learn, the better you become in learning and dealing with more challenging subjects.
Language skills, learned to read, speak AND write:
- Arabic (USC — adult learning, night classes)
- Russian (self-taught)
- Japanese (BC college, as an older student — not in my 20’s)
- Chinese (UW university, older student)
Music skills, learned to read Treble and Bass scale music to play:
- Trumpet (high school marching band/concert band)
- Piano (college, older student)
Advanced Math in college
Physics (the version with calculus — many folks take the ‘non-calculus’ version of Physics)
Chemistry
Now, let me get the ‘possibly’ bad bullet out of the way:
1) Race — MOST of the time, this is difficult to determine
Then, there is the:
2) requirement of having 2, 3–10 years of ‘hands-on DE work experience’
- As a solid viable substitute, does it no longer hold water for achievements such as:
- — — holding double DE certifications
3) Salary
- my pay has been high for years but I am still willing to take a pay cut to get my foot in the Data Engineer door and have made that clear as well, just not too drastic of a pay cut — not with all the pluses in my basket.
So, in our current time period in the workplace, “does drive and sacrifice no longer hold any water?”
Especially for anyone who has stated to recruiters and the various companies that one is willing to come in at mid-level (in my case, it could not be junior level due to my years of AWS cloud experience and the degrees and the certifications) and do the grunt work for the first 6–8 months…?
What are recruiters and companies looking for? One can only do so much studying, along with some projects (I’ve read a couple of articles where companies do not care about the projects you did) prior to being hired. Especially when they need to return to work.
If a person’s drive and sacrifice is not a completely solid indicator that they want to do that work, what else could be a better indicator to bring that person in…?
This is a very interesting tech world we live in now. For the others of you having similarly tough progress, good luck in your journey.
.
.
Jul 2024 Update
At this point, I did have an offer which was retracted due to their fear that I might walk away from them again. In 2022, I had walked from them for a Cloud Architect role at another company for that same role at a much higher salary.
- Retracted offer, even though I told the company that this time I would not walk away again… Because of the time lost and savings burnt at this point and their retraction, I would now, never work for them again and steer colleagues/friends away from that company.
Sept 2024 Update
But, I did receive another offer from a company which I accepted as a Data Engineer, at a location 10 minutes away — but their process is SOOOOO BLOODY LONG. It does not start until late Sept (due to security clearances updates and whatnot).
So, before work starts, my goals are to continue STUDYING MY BUTT OFF EVERY SINGLE DAY:
- SQL & Python
- Data Engineering
- ML Engineering
I will guarantee in making myself valuable to any/every company who would want me. That is something you all out there had best work on, trying to figure out things on your own as much as possible — research. Besides, doing that research pays off in you covering additional grounds at the same time for tangential areas…
.
tags: acloudguru, aws certified data engineer, cloudacademy, data engineer, data engineering essentials, gcp pro data engineer, google cloud platform data engineer, aws ml specialist, aws machine learning specialist, drive, sacrifice, drive and sacrifice, cognitive diversity, communication skills, diplomacy skills, aws cloud, Jupyter Notebook, Thonny, VScode, VS Code, pgAdmin, Pandas, Numpy, ML, Machine Learning
