IT/Software career thread: Invert binary trees for dollars.

Noodleface

A Mod Real Quick
40,577
20,655
I'm still working through a significant backlog from the merger. None of our testing infrastructure worked with theirs, so I'm having to reimplement most of our test suites. We just got attachments back last week, after more than a year without them. Imagine diagnosing failures with the relevant logs missing(AI was actually quite useful here).

But, it's probably myopic to blame just that. Engineers suck at estimating task effort and I'm no exception. We've actually given up on assigning any estimate. So planning is just this kind of "yeah, that'll fit" guessing game.
Ah yeah I think this is probably part of the problem. Also sounds like you're working off hours too - what if you just stop doing that?
 

Deathwing

<Bronze Donator>
17,860
8,797
Ah yeah I think this is probably part of the problem. Also sounds like you're working off hours too - what if you just stop doing that?
I honestly don't know. It's a fair question and I don't know why I'm not more insistent about sticking to regular working hours despite being sick of it. I've become more non-confrontational as I've aged and I don't like letting others down. Off hours work is often catching up on stuff I feel is blocking others, like reviewing MRs. AI has made that worse recently.
 

Control

Avatar of War Slayer
6,439
17,849
Yes, sometime during standup, but often during weekly meetings.
as much as i loath meetings, especially in the office, this is one case where a big whiteboard with all of your sprint postits comes in handy.
"great! i'll get your 10 new postits added right now! which ten current ones should move to the "these get fucked" stack?"
 

Noodleface

A Mod Real Quick
40,577
20,655
I honestly don't know. It's a fair question and I don't know why I'm not more insistent about sticking to regular working hours despite being sick of it. I've become more non-confrontational as I've aged and I don't like letting others down. Off hours work is often catching up on stuff I feel is blocking others, like reviewing MRs. AI has made that worse recently.
Not judging as I've been in some serious crunches - but does anyone else work this much? Your boss maybe?

In my opinion, just based on what you said, your team really needs better sprint planning and as Control Control said you need to move less important things out if they're coming in like that. There are always going to be spikes, but there needs to be some give and take with that. When we take a spike, we move something else completely out of the spring unless we think it's a story that someone else can swarm on. Now... I hate scrum terms, but it's the easiest way to describe what I mean.
 
  • 1Like
Reactions: 1 user

TJT

Mr. Poopybutthole
<Gold Donor>
48,104
132,741
I honestly don't know. It's a fair question and I don't know why I'm not more insistent about sticking to regular working hours despite being sick of it. I've become more non-confrontational as I've aged and I don't like letting others down. Off hours work is often catching up on stuff I feel is blocking others, like reviewing MRs. AI has made that worse recently.
While I do understand not letting others down its just a corporate job.

What I mean here is that as you have been around at that job for a long time you should have leverage. I've gained enough expertise, leverage, and confidence at this point that I shutdown at 4PM and stop responding to messages entirely. No way they would ever fire me or anything as I complete 3x the work of anyone else anyway. If you are clearing all your sprint work don't just go and do extra, force leadership to make tickets, document stuff, and so on.
 
  • 4Like
Reactions: 3 users

Kuro

Bronze Baron of the Realm
11,299
36,557
Looks like I am IT Director for the next 90 days just without the pay or title.

For my first act of cruel tyranny, I will force everyone over 60 to use New Outlook
 
  • 5Worf
  • 2Mother of God
  • 1Like
Reactions: 7 users

Seananigans

Honorary Shit-PhD
<Gold Donor>
16,272
38,358
Looks like I am IT Director for the next 90 days just without the pay or title.

For my first act of cruel tyranny, I will force everyone over 60 to use New Outlook

Make sure everyone needs a new password with 4 special characters every 2 days
 
  • 2Like
Reactions: 1 users

Noodleface

A Mod Real Quick
40,577
20,655
Looks like I am IT Director for the next 90 days just without the pay or title.

For my first act of cruel tyranny, I will force everyone over 60 to use New Outlook
Just 90 days you say?

Don't do too well of a job or without the pay or title will be permanent.
 
  • 2Like
  • 1Truth!
Reactions: 2 users

Control

Avatar of War Slayer
6,439
17,849
Just 90 days you say?

Don't do too well of a job or without the pay or title will be permanent.
Well established practice, you gotta lock the good stuff behind a paywall!
1792528650595.png
 
  • 1Worf
  • 1Like
Reactions: 1 users

Kuro

Bronze Baron of the Realm
11,299
36,557
DNS server down this morning, and of course salespeople came in on a Saturday when they're not supposed to be here so they're all panicking that they can't reach the file server. Dragged my sick ass out of bed to drive down here and get it fixed.

EDIT: Took a while but figured out the Virtual Servers for the DNS and backup DNS had somehow rolled over their memory limit at 2:30 in the morning. Was able to increase partition size and get them back up, but need to figure out how that happened once the work week starts.
 
Last edited:

Kuro

Bronze Baron of the Realm
11,299
36,557
We have a couple hundred arm-crusher-9000 machines connected to the network, including routing (cutting) machines that assign themselves an internal IP that you can only see in their bespoke app but are technically still on the network and if you try to connect to their IP they turn on and chop the hands off of anyone trying to position metal sheets on the table at the time.

This setup was inherited from the previous IT team.
 
Last edited:
  • 1Like
  • 1Worf
Reactions: 1 users

TJT

Mr. Poopybutthole
<Gold Donor>
48,104
132,741
I got one for TomServo TomServo

So again, I work in data infrastructure so this is more side drama to me. A few weeks ago a B2B sales guy comes to me and asks how we expose data in general. I said that that data platform itself is very much locked down. It does not export any of our internal data directly to anyone outside of certain services like a few financial ones we have. This guy was not really technical but he was very astute and he explains to me what he has been seeing.

In a several of his sales calls he said the potential customer knew far more than he thought they would about our sales process, final prices, and so on. I spend like 30 minutes showing him some of the inflows and outflows of this data, where it sits physically on our on prem systems and what consumes it. The primary consumer being the product application itself. The one that drives the entire business. He just goes okay, I will continue looking.

He goes to the app devs asking about it and this makes the rounds as the tech org is all seated in the same area of this startup type company. One that is doing ten times more business than it was 4 years ago and we have billions in revenue now. The app dev team finally asks this guy for some examples of what he suspects. Sure enough the customers are using datapoints that app dev says they should absolutely not have access to. Now we are forced on the hunt and to find it. Very easy to prove it can't be my space so we move on from that quickly.

Keep in mind we have exactly one dedicated infosec guy in the entire company and he is much more of a traditional IT systems infosec guy. He knows almost nothing about application architecture and security in general. The cause? Our internal only product was not nearly locked down as we thought (lol). The rotating key system that did indeed rotate internal access keys per session was actually exposing them in plaintext and the internal app itself not fully isolated from the greater internet. Leading to some of our customers figuring this out and scraping it constantly.

Our industry is highly transactional and volatile making this data very helpful when it comes to negotiating pricing and they knew what all of their competitors were paying. Overall a small sample of customers figured this out but still. LOL. The general philosophy was that the teams can handle their own infosec stuff. I bet we get a few hires for it now.
 
  • 3Like
  • 1Double Worf
  • 1Worf
Reactions: 4 users

TomServo

<Bronze Donator>
9,064
23,676
I got one for TomServo TomServo

So again, I work in data infrastructure so this is more side drama to me. A few weeks ago a B2B sales guy comes to me and asks how we expose data in general. I said that that data platform itself is very much locked down. It does not export any of our internal data directly to anyone outside of certain services like a few financial ones we have. This guy was not really technical but he was very astute and he explains to me what he has been seeing.

In a several of his sales calls he said the potential customer knew far more than he thought they would about our sales process, final prices, and so on. I spend like 30 minutes showing him some of the inflows and outflows of this data, where it sits physically on our on prem systems and what consumes it. The primary consumer being the product application itself. The one that drives the entire business. He just goes okay, I will continue looking.

He goes to the app devs asking about it and this makes the rounds as the tech org is all seated in the same area of this startup type company. One that is doing ten times more business than it was 4 years ago and we have billions in revenue now. The app dev team finally asks this guy for some examples of what he suspects. Sure enough the customers are using datapoints that app dev says they should absolutely not have access to. Now we are forced on the hunt and to find it. Very easy to prove it can't be my space so we move on from that quickly.

Keep in mind we have exactly one dedicated infosec guy in the entire company and he is much more of a traditional IT systems infosec guy. He knows almost nothing about application architecture and security in general. The cause? Our internal only product was not nearly locked down as we thought (lol). The rotating key system that did indeed rotate internal access keys per session was actually exposing them in plaintext and the internal app itself not fully isolated from the greater internet. Leading to some of our customers figuring this out and scraping it constantly.

Our industry is highly transactional and volatile making this data very helpful when it comes to negotiating pricing and they knew what all of their competitors were paying. Overall a small sample of customers figured this out but still. LOL. The general philosophy was that the teams can handle their own infosec stuff. I bet we get a few hires for it now.
Can i come work there? No homo. A serious response tomorrow.

I spent the day losing my shit on people and i got no chill right now
 
  • 1Solidarity
Reactions: 1 user

Sheriff Cad

scientia potentia est
<Nazi Janitors>
35,778
86,914
I got one for TomServo TomServo

So again, I work in data infrastructure so this is more side drama to me. A few weeks ago a B2B sales guy comes to me and asks how we expose data in general. I said that that data platform itself is very much locked down. It does not export any of our internal data directly to anyone outside of certain services like a few financial ones we have. This guy was not really technical but he was very astute and he explains to me what he has been seeing.

In a several of his sales calls he said the potential customer knew far more than he thought they would about our sales process, final prices, and so on. I spend like 30 minutes showing him some of the inflows and outflows of this data, where it sits physically on our on prem systems and what consumes it. The primary consumer being the product application itself. The one that drives the entire business. He just goes okay, I will continue looking.

He goes to the app devs asking about it and this makes the rounds as the tech org is all seated in the same area of this startup type company. One that is doing ten times more business than it was 4 years ago and we have billions in revenue now. The app dev team finally asks this guy for some examples of what he suspects. Sure enough the customers are using datapoints that app dev says they should absolutely not have access to. Now we are forced on the hunt and to find it. Very easy to prove it can't be my space so we move on from that quickly.

Keep in mind we have exactly one dedicated infosec guy in the entire company and he is much more of a traditional IT systems infosec guy. He knows almost nothing about application architecture and security in general. The cause? Our internal only product was not nearly locked down as we thought (lol). The rotating key system that did indeed rotate internal access keys per session was actually exposing them in plaintext and the internal app itself not fully isolated from the greater internet. Leading to some of our customers figuring this out and scraping it constantly.

Our industry is highly transactional and volatile making this data very helpful when it comes to negotiating pricing and they knew what all of their competitors were paying. Overall a small sample of customers figured this out but still. LOL. The general philosophy was that the teams can handle their own infosec stuff. I bet we get a few hires for it now.
Do you think any of the customers have discovered that other customers were scraping their payments data?

The risk is going to be that when you close the loophole the guys who were scraping it throw a fit and expose that they were getting it (I know it makes them look bad, people don't care) then the other customers find out and your company gets sued for exposing data.
 

TJT

Mr. Poopybutthole
<Gold Donor>
48,104
132,741
Do you think any of the customers have discovered that other customers were scraping their payments data?

The risk is going to be that when you close the loophole the guys who were scraping it throw a fit and expose that they were getting it (I know it makes them look bad, people don't care) then the other customers find out and your company gets sued for exposing data.
Most of our customers are not tech companies so I don't think so. The overall pricing of this industry is heavily subjective to seasonal factors anyway so it was not some catastrophic thing. Just the result of years of rapid business growth and the dev org just keeping their heads above water. What it did allow is for the companies doing it to be able to say "we know other customers are paying X in this situation, why aren't we getting closer to that?" This put sales on the backfoot.

Obviously this is still not desirable so engineering will just dedicate more resources to security, which they have never done before. My old job was also a later stage startup and they had an Application Security Architect and an Application Architect just for the product itself. In addition to the normal InfoSec guys who were mostly retards. Overall they were just playing a lot faster and looser here. It shows in various places.
 
  • 2Like
Reactions: 1 users

Noodleface

A Mod Real Quick
40,577
20,655
Most of our customers are not tech companies so I don't think so. The overall pricing of this industry is heavily subjective to seasonal factors anyway so it was not some catastrophic thing. Just the result of years of rapid business growth and the dev org just keeping their heads above water. What it did allow is for the companies doing it to be able to say "we know other customers are paying X in this situation, why aren't we getting closer to that?" This put sales on the backfoot.

Obviously this is still not desirable so engineering will just dedicate more resources to security, which they have never done before. My old job was also a later stage startup and they had an Application Security Architect and an Application Architect just for the product itself. In addition to the normal InfoSec guys who were mostly retards. Overall they were just playing a lot faster and looser here. It shows in various places.
The fun part is now non tech companies can throw Claude at it and find out all that stuff in 5 minutes
 
  • 2Like
Reactions: 1 users

Noodleface

A Mod Real Quick
40,577
20,655
Getting so monumentally burnt out recently. It's been coming for a couple years I guess but even more so recently.

I've mentioned it but I am the tech lead for a very large piece of a business critical product. For several generations we were a nice commodity that was supported in a limited number of SKUs, but this generation our support expanded ten-fold (or more, they keep adding SKUs). About 90% of our code is common, but the last 10% can't be due to CPU/chipset differences and hardware differences. The absolute braindead PMs don't talk to each other so they scheduled all of these on top of each other. But that's only a part of the problem really. .
The bigger problem is no one in the company seems to know how to debug problems or get the right people involved. I feel like I am the only person that actually understands the product, what the error conditions translate to, and who should own the fix. Sometimes it's my team, but a lot of times it's not. And debugging this product is very complicated because it's an end-to-end product with pieces in all aspects of the hardware and software. Valuable skill for me, but I'm absolutely drowning.

Even just today I asked a team member to take a ticket because I am overrun dealing with issues. They just floundered and couldn't even be bothered to read release notes to check when a certain fix had been introduced. Threw their hands up in the ticket and told the reporter some bullshit that was way off base. So in the end I just said fuck it give it to me, and I transferred the ticket to our sustaining team. Meanwhile the dude is throwing up constant code reviews made by Claude that I simply do not have enough time to handle.

Even then .. the changes the team makes are so low impact. I asked another engineer if we could investigate some huge improvement I thought of in my head, a new architecture for our stuff that I think could solve many issues we saw. They simply said it can't be done. So I said fuck this, implemented it myself (and got it patented), got business unit buy in and got praised for such a great idea.

I know a lot of people think without them a company will fail, but this is the first time in my career I've felt both completely overwhelmed and drowning and also uniquely in a position where without me we'd actually lose business.

I don't know what the solution is here, so I'm mostly bitching. Had to write this shit down. I have a 1x1 with my manager next week where I need to at least make him aware. I actively feel like I'm drowning.