r/sysadmin • • 4d ago

General Discussion Weekly 'I made a useful thing' Thread - October 23, 2026

2 Upvotes

There is a great deal of user-generated content out there, from scripts and software to tutorials and videos, but we've generally tried to keep that off of the front page due to the volume and as a result of community feedback. There's also a great deal of content out there that violates our advertising/promotion rule, from scripts and software to tutorials and videos.

We have received a number of requests for exemptions to the rule, and rather than allowing the front page to get consumed, we thought we'd try a weekly thread that allows for that kind of content. We don't have a catchy name for it yet, so please let us know if you have any ideas!

In this thread, feel free to show us your pet project, YouTube videos, blog posts, or whatever else you may have and share it with the community. Commercial advertisements, affiliate links, or links that appear to be monetization-grabs will still be removed.


r/sysadmin • • 13d ago

General Discussion Patch Tuesday Megathread - (October 13, 2026)

111 Upvotes

Hello r/sysadmin, I'm u/AutoModerator, and welcome to this month's Patch Megathread!

This is the (mostly) safe location to talk about the latest patches, updates, and releases. We put this thread into place to help gather all the information about this month's updates: What is fixed, what broke, what got released and should have been caught in QA, etc. We do this both to keep clutter out of the subreddit, and provide you, the dear reader, a singular resource to read.

For those of you who wish to review prior Megathreads, you can do so here.

While this thread is timed to coincide with Microsoft's Patch Tuesday, feel free to discuss any patches, updates, and releases, regardless of the company or product. NOTE: This thread is usually posted before the release of Microsoft's updates, which are scheduled to come out at 5:00PM UTC.

Remember the rules of safe patching:

  • Deploy to a test/dev environment before prod.
  • Deploy to a pilot/test group before the whole org.
  • Have a plan to roll back if something doesn't work.
  • Test, test, and test!

r/sysadmin • • 16h ago

Moment of Solidarity for those who have had to terminate a Co-worker or Friend

742 Upvotes

Today, I got the news that a co-worker of mine passed away. She was one of the first people I on-boarded at this client when I previously worked for them as a tech with an MSP and not as their manager. She was always smiling and made me feel so accepted at a place that I was unfamiliar with. She was a mother, under 50 yrs old and had a young daughter she was co parenting. I would get to see her daughter often at the org and you could tell how much her mom loved her.

Now, she is gone and I have to retrieve her laptop. Maintenance has to clean up the office and we have to move on.

It is the second person this year that I have closely worked with that I will have to terminate and it is devastating.

I know this is a somewhat common post here, but I think it's a conversation that is very important for those of us that have had to deal with grief in the workplace.

She had a direct impact on my experience at work and me feeling like I was at the right place in my career.

I want her memory to live on so I will tell others of our interactions and how sweet she was.

If you have had to go through similar, please feel free to share your own experiences with this.
Sending love to anyone dealing with death.


r/sysadmin • • 5h ago

Out-of-band .NET patches landed Oct 25 for the Brotli CVE — fixed builds are 8.0.33 / 9.0.22 / 10.0.14, and PowerShell got fixes too

56 Upvotes

Microsoft shipped .NET updates out of band on Saturday (Oct 25) for CVE-2026-111497, an out-of-bounds write in the Brotli decoder bundled with .NET. The .NET advisory scores it 8.1 and MSRC rates it Critical for .NET, and Microsoft says exploitation is more likely, with no exploitation reported as of the 25th.

The patched runtimes are 8.0.33, 9.0.22 and 10.0.14. Anything below those is still exposed, and .NET Framework is not affected, so older Framework servers can sit this one out. The same release also fixes CVE-2026-114148, a file-ownership problem in .NET tarballs on Linux, and it re-does two October 13 fixes that had not fully landed (CVE-2026-70197 and CVE-2026-102870).

A few things that decide how urgent this is on a given box:

Exposure needs an app that decompresses Brotli data it does not control, through ASP.NET Core request decompression, HttpClient automatic decompression, or the System.IO.Compression Brotli APIs. Neither decompression path is on by default, so a patched-later queue is defensible for internal apps that never enable them, while internet-facing ASP.NET Core goes first.

Updating the shared runtime does nothing for self-contained apps because they carry their own copy of the runtime, and those need a rebuild and redeploy. Framework-dependent apps pick the fix up from the runtime update, though the processes still need a restart, and on IIS that means updating the ASP.NET Core Hosting Bundle rather than only the runtime.

PowerShell is in scope as well, with fixes in 7.4.22, 7.5.13 and 7.6.9, which matters on admin workstations that never run a .NET app. Anyone on Azure App Service with built-in .NET images on Linux gets the platform update from Microsoft; the rollout had finished for the Latest channel and was still going out to Standard as of Oct 26.

`dotnet --list-runtimes` gives a quick inventory per machine. Microsoft's announcement is here: https://devblogs.microsoft.com/dotnet/dotnet-october-2026-oob-security-updates/


r/sysadmin • • 16h ago

Rant Get ready for more slop from Microslop

277 Upvotes

As if Microsoft and their software/cloud isn't bad enough in it's current state, it's likely going to get worse with a reduction of about 1/4 of their workforce.

Microsoft makes new 'voluntary exit' offer to about a quarter of its workforce, including some execs – GeekWire

I'm eager to see how many more shit updates get through MS "QA" into production environments and cause unnecessary havoc and stress globally.


r/sysadmin • • 1h ago

Police Evidence

• Upvotes

We were sadly compromised. Twice. It's finally answered the question of investments, but the police want evidence.

The audit logs is clear on that, but I realise this will become police evidence. They have a secure upload. I filtered the audit logs as I downloaded them to show everything from the attackers to create the csv file. I then created a text file showing the filter, how it was created and the SHA hash of the file.

Is this enough? I think the attacker(s) made a mistake and may be traceable, so absolutely would be willing to see them prosecuted.

The question is, should I have included anything else in there or done anything else to show the file is not tampered with between myself creating and them opening?


r/sysadmin • • 6h ago

General Discussion our bot rules weren't built for AI agents that buy

36 Upvotes

Walmart opened its store to AI shopping agents last week while Amazon is blocking Meta's, and it landed right as we're finishing our Black Friday WAF rules.

Every year our checkout rules get stricter going into peak, anything that looks automated gets challenged or blocked, and that's done a good job against card testing and credential stuffing.

An agent buying for a customer looks a lot like those bots from our side, headless browser or straight API calls, fast and with no mouse movement, and from next month some of that traffic will be real orders.

Our shop is headless (Scayle), so search, cart and checkout are already APIs an agent can call, which means the whole decision lands on the WAF.

The business side hasn't decided whether we copy Walmart, copy Amazon or let a few known agents through, and the rules freeze on Nov 1.

So before I pick a default myself, how are you treating agent traffic going into Black Friday ?


r/sysadmin • • 51m ago

Acquired company, their one IT guy quit and we have no experience with QNAP: fw upgrade for TS-431XeU?

• Upvotes

We acquired a company that uses QNAP NAS'es but we have no experience with these whatsoever. Defender is telling me to update our TS-431XeU because of the recent PHP2Shell vulnerabilities, but looking at the details, it's still running QTS 4.3.4.0675 , so we're about eight fucking years behind.

However, checking for updates through QTS says the system is up-to-date while the QNAP support page lists newer firmware for this model. I assume we'll have to update it manually, but I'm not sure whether we can jump straight to QTS 5.x or if we need to install intermediate versions first.

My question: is it safe to update across such a large version gap and what does the process look like? Any indications of update duration, possibilities of failure, compatibility issues with existing RAID configurations or volumes, etc.? Also, what are the recovery options if the firmware update fails or the NAS doesn't boot afterwards? Fuck me but also fuck that guy for leaving no documentation.

We will of course back up the data but I'd rather not risk anything since this is a production NAS and we don't know much about its history.

Anyone who's done a similar upgrade on this model or another older QNAP?

Edit: fuck it, I created a ticket at QNAP but since they're not reachable from outside, I'll just put a wall around it as someone said here. Thanks for the info, my dudes


r/sysadmin • • 16h ago

General Discussion PHP2Shell just made the week hell. AI is going to make this our new normal.

165 Upvotes

Is anyone else's team still recovering from the recent PHP2Shell fire drill (CVE-2026-109618 / 109619)?

How we got here is what scares me the most. The developers knew about a memory issue back in 2024. They ignored it to keep older systems working, thinking no one could actually use it to hack into systems.

But then, someone used a new AI model on the old C codebase. The AI quickly found a complex bug that humans had missed for years. To make things worse, a software company accidentally released the fix 12 hours too early. Because of this mistake, hackers could see exactly what was changed. They started attacking the internet before the official warning even came out.

For 20 years, old open-source code stayed safe mainly because finding memory bugs by hand is incredibly hard, boring work. Now, AI models can easily find hidden flaws in 15-year-old code. Volunteer developers simply can't keep up with this speed.

How are you all handling this at work? If we now have to fix servers in four hours or unplug them completely, I might just move to the woods and raise goats.


r/sysadmin • • 10h ago

Is a $17 a hr onsite hardware support job worth taking to break into IT?

37 Upvotes

I'm trying to transition into IT and would appreciate some advice from people who have made a similar move.

I have an upcoming interview for a Customer/Hardware Specialist position at a dental imaging company. It's a six-month contract-to-hire role paying $17/hr, onsite, Monday through Friday.

The job involves Level I support calls and emails, troubleshooting hardware and software issues, working with medical imaging equipment, installing equipment, training customers, documenting issues, and escalating problems when needed. My background is in education technology, where I've supported devices, printers, software, and end users, but I haven't worked in a dedicated IT support role.

I'm planning to work toward CompTIA certification and stack others.

My situation is that I have about 2.5 months of unemployment benefits remaining at $584 a week, and I'm currently living with family without paying rent. At $17/hr for 40 hours, the job would pay $680/week before taxes, so the financial difference isn't huge.

I'm trying to decide whether to take an opportunity like this to get actual IT experience while studying for certifications, or use the remaining time to study and look for a remote job paying around the same amount or more.

I think the hands-on experience could help me qualify for better IT jobs down the road. Another concern is that a six-month contract could limit my flexibility if a better opportunity comes along, and working onsite might make it harder to study.

Thanks in advance for any insights you guys have!


r/sysadmin • • 22h ago

I'm so stressed working in MSP

285 Upvotes

I just wanted to get this off my chest. I don't have anyone to talk to who would understand what I'm going through.

I'm in hell. The metrics are unattainable, and there are too many tickets. I'm getting anxious every day, and I've even started dreaming about tickets. My managers call me out almost every week because I miss a ticket.

I want to leave this job, but I can't for now.


r/sysadmin • • 15h ago

General Discussion I feel like I'm trapped on the Ship of Theseus...

58 Upvotes

... and rats are gnawing the floorboards.

(To be kinda meta, this post isn't seeking anything except commiseration and conversation. I'm just reflecting on the dizzying array of changes within the last ~3 years, and hoping a graybeard out there can relate to feeling ... I don't know, displaced? surreal? anyways, my apologies if this doesn't belong in r/sysadmin*)...*

While the underlying tech has evolved, the core of the Internet and computing remains the same as it was in the 1980s: content is stored on hosts and delivered to clients through multiple layers of protocols.

It makes sense, then, that AI will discover uncontrollable amounts of vulnerabilities. The visionaries who initially drafted the RFCs for DNS, HTTP, and TCP/IP couldn't have possibly imagined our data getting commodified, names and birthdays sold for pennies or held for ransom, financial ruin inevitable. Of course they baked their naiveties into the tech. Of course history is getting sandwiched between OpenAI and Anthropic, with attendant freakouts from suits and normies alike.

It seems like the wooden planks are decaying, but the replacements don't fit anywhere yet. Maybe it's time we transition away from the existing infra and replace it in its entirety with the Semantic web and MCP/agents. Give up the whole ghost.

Can anyone relate? Does this resonate at all?


r/sysadmin • • 9h ago

Microsoft Defender causing high CPU usage on Windows servers

12 Upvotes

​

Hi everyone,

I'm seeing intermittent high CPU usage from Microsoft Defender Antivirus ("MsMpEng.exe"). CPU usage reaches around 90–200%, and the alerts usually resolve themselves after a few hours.

No active scan was running during troubleshooting, and Defender appears healthy. I suspect real-time scanning triggered by application file activity may be involved.

Has anyone experienced something similar? What would you recommend checking first to identify the root cause?

Thanks!


r/sysadmin • • 13h ago

Question Is Jamf necessary for an environment with less than 10 Mac computers and no other mobile devices?

21 Upvotes

I'm very experienced in Intune, don't know much Jamf yet, so this is a new platform for me. Just wondering if it's beneficial to use it in an environment that only has a handful of Macbooks?


r/sysadmin • • 18h ago

Microsoft Teams Issues

45 Upvotes

Anyone else seeing issues with Microsoft Teams this morning? Messages not being sent or received?

Edit: Thanks for confirming everyone. GCC here as well. Tested from external cell phone as well to make sure it wasn't our internal network but wasn't seeing anything on the M365 service health page.


r/sysadmin • • 10h ago

SysAdmin Hesitancy

9 Upvotes

I've been an IT generalist at a large tech company for a few years and I will be "next in line" for a SysAdmin role soon, however, I don't think the stress and responsibility is worth the reward and opportunity.

I clearly am complacent on some level, but I am still challenged enough in my day to day that I stay engaged and interested while working. I make ~$115k/year gross including bonus, with ~5% raise yearly. I am in HCOL area but feel I am compensated well. I like my team and I like my company, which I think is rare.

In my late 30's, have bachelors in IT, almost 10 years in the field, a couple of basic IT and networking certs. I am married and don't plan to have kids. I have hobbies I am passionate about, but I have zero interest in a home lab or spending free time learning new IT skills. The extra income would be nice, of course, but the trade-off's are real, and unfortunately I don't think I can go back to my current role here if I make the jump. Pressure from management to move up is real.

I'm sure folks can relate - what do you think?


r/sysadmin • • 1h ago

Is windows 2025 finally production ready?

• Upvotes

I have to setup a new domain and I am strongly considering windows 2025 for smb without VPN and to save myself from reinstalling the DC two years from now. Given the frequent complains here I wanted to circle back with you people if anybody having a recent update roll up installed is facing problems? Especially with the RDP broker and trust issues between servers (also domain joined Linux VMs).

The domain would be a pure 2025 AD with roughly 50 clients providing file access and DNS, dhcp handled by pfsense. Authentication would be provided to web apps through privacyIdea and keycloak.

Fire away!


r/sysadmin • • 1d ago

if your blacklist checker uses Google or Cloudflare DNS, Spamhaus results are probably wrong

162 Upvotes

Ran into this while chasing a bug report and it surprised me, so sharing.

Spamhaus doesn't answer queries that come through big public resolvers. Google returns an empty answer, which your software reads as "not listed". So a Postfix reject_rbl_client zen.spamhaus.org on a box that resolves through 8.8.8.8 lets everything through, and nothing ever logs an error.

10-second test, run on your mail server:

dig +short 2.0.0.127.zen.spamhaus.org

That's Spamhaus's permanent test entry; it should always return 127.0.0.2. Empty or 127.255.255.x means your lookups aren't working.

Fix: run a local recursive resolver (Unbound takes five minutes) and point the mail server at it. If you're non-commercial, a free Spamhaus DQS key also works.

How I found it: I maintain a small self-hosted blacklist monitor, and a user reported an IP that was on Spamhaus but showed clean. Same cause. It now tests every list against its test address first and shows "no answer" instead of a fake "clean".

Curious how common this is: what does the dig return for you?


r/sysadmin • • 3h ago

General Discussion EpmapSavedContextCap=65536 workaround for the October 2026 Exchange issue – still stable? How do you monitor it?

1 Upvotes

For those who set HKLM\SOFTWARE\Microsoft\Rpc\EpmapSavedContextCap = 65536 on Exchange instead of uninstalling the October KB (https://techcommunity.microsoft.com/blog/exchange/exchange-servers-may-become-unstable-after-installation-of-october-2026-windows-/4565382): are your servers stable since then, or are you still seeing MdbOffline / mail flow lockups? Any side effects, and are you doing scheduled reboots until a real fix ships?

Also, how are you monitoring Exchange for this issue?


r/sysadmin • • 15h ago

Question DHCP migration 2019 -> 2025: Import-DhcpServer fails with "Network Access Protection is not supported by the server (20138)"

5 Upvotes

Hi all,

I'm migrating a Windows Server 2019 DHCP server (~200 scopes) to a new Windows Server 2025 server.

I exported with Export-DhcpServer -Leases on the 2019 server, but Import-DhcpServer on 2025 fails with:

"Failed to set server configuration on DHCP server ... Network Access Protection is not supported by the server. (20138)"

NAP was never used on the old server, but the export XML still contains NapEnabled / NpsUnreachableAction (server level) and NapEnable / NapProfile (scope level). Removing these elements from the XML breaks the schema validation ("The element 'IPv4' has invalid child element 'ActivatePolicies'. Expected: 'NapEnabled'").

Options I'm considering:

  1. Import scope by scope with -ScopeId in a loop and move the server-level settings (option definitions, classes, server options, policies, filters) separately.

  2. Skip the XML entirely: create a DHCP failover relationship from the 2019 server to the 2025 server with all scopes, let it replicate, then remove the relationship.

Questions:

- Has anyone hit this? Which approach did you use and did anything get lost (server options, vendor classes, policies, DNS credentials)?

- Is there a supported way to make Import-DhcpServer ignore the NAP settings?

- Any gotchas with the failover method for ~200 scopes (time, lease state, reservations)?

Thanks!


r/sysadmin • • 12h ago

General Discussion How do you notice a host froze halfway through a long SSH command?

4 Upvotes

ssh sends no keepalives of its own by default (ServerAliveInterval 0), and Linux waits two idle hours before the first TCP probe, so a script on a frozen host can hang for hours. Set the interval to 15 and a 45-second network blip kills the session. Do you tune keepalives, run commands detached and poll, or wrap them in timeout?


r/sysadmin • • 17h ago

Question "Something went wrong" - Logging onto Microsoft Apps

9 Upvotes

We are seeing a increase reports of an error "Something went wrong" after logging onto any Microsoft app. Windows 11 26H2 running the latest Windows Updates.

Found this post where it was suggested KB5127648 was the cause.

Anyone else?


r/sysadmin • • 12h ago

Backup-to-Tape Infrastructure and Job Design

2 Upvotes

Hello everyone,

We are planning to expand our current tape infrastructure by adding an expansion module to our existing LTO-8 tape library.

Our current setup consists of 2 tape drives and 40 slots. After the expansion, we will have a total of 4 tape drives and 80 slots. The tape drives use Fibre Channel (FC) to transfer data from the primary server.

Reason

Our RTO and RPO have increased significantly, as we are now transferring around 200 TB of data per week. We need to improve them.

I also have the feeling that our current logical infrastructure design may not be the most efficient. We are currently using Veeam Backup & Replication to perform our backup-to-tape operations.

Current

We currently have a single Standard Media Pool containing all our tapes. None of them are WORM tapes.

We use a single tape job with no incremental backups, which copies only the weekly full backups to tape. Parallel processing is enabled in the job, allowing it to use both LTO-8 drives simultaneously. The tapes are labelled both logically and physically according to their retention category: weekly, monthly, and yearly. We also label each tape with a DIMO. We do not use quarterly tapes.

Our current process is entirely manual. We select and insert different tapes depending on the occasion, rather than having an automated process. To be honest, I really dislike our current labelling system. All tapes have the same retention settings in Veeam. We manage retention manually by moving the tapes and storing them physically for the required period.

Proposed

Since we will have 80 slots available, I was thinking about using the additional capacity to keep the weekly tapes and, potentially, future daily tapes loaded in the library, as these will be the tapes we use most frequently. Our weekly tape rotation requires a total of 64 tapes, so this would allow us to keep them in the library instead of having to move around 16 weekly tapes to our off-site vault.

However, this raises my first question: Is it recommended to leave tapes loaded in the library when they are not actively being used?

I understand that these are not WORM tapes and that, in the event of a cyberattack, they could potentially be exposed to overwriting. One idea I had was to use the physical write-protect switch on the tapes to prevent them from being overwritten while they are idle.

On the other hand, I have also considered the risk of a physical incident in the data centre, although we consider the likelihood of a fire or flood to be relatively low.

Is this a reasonable approach, or is it strongly recommended to remove written tapes from the library and store them in an off-site vault?

For those managing larger tape libraries with many slots, how do you typically handle this? Do you leave tapes stored in the library and use the available slots as intended, or do you still remove written tapes manually, as we currently do?
From a Veeam configuration perspective, I am considering the following changes:

  • Media Pool: Use a single GFS Media Pool containing all retention types, excluding quarterly tapes.
  • Tape labelling: Eliminate our current logical and physical labelling system and rely entirely on barcodes to identify the tapes.
  • Tape job configuration: Keep parallel processing enabled and configure the job to use all four tape drives.
  • Media Vault: Set up a Media Vault in Veeam to better organise and track the tapes stored outside the library, as I understand this functionality is available in Veeam.

These changes would be implemented progressively as our tape operations evolve, rather than all at once.

Do you have any recommendations regarding this approach? Are there any potential pitfalls or configuration changes that you would advise against?

Thanks!!


r/sysadmin • • 8h ago

Question Delivery tracking with SharePoint?

2 Upvotes

So I got tasked with looking into a solution for keeping track of deliveries that come in. I used chatgpt & did some research and I found a couple paid options but chatgpt suggested using SharePoint + Power Automate + Microsoft lists. I started looking into that as a solution and made a SharePoint site with a test list for myself. However the CTO would like the system to send an email to whomever is having a package delivered. So the front desk would log the delivery in SharePoint. I'm a little lost on how to trigger the sending of an email to the package receiptent. Is that where power automate comes into play?

Does this plan sound feasible at all? I'm pretty new to SharePoint.


r/sysadmin • • 1d ago

General Discussion Chrome Shows “Not Secure” After SSL Certificate Update

60 Upvotes

The SSL certificate has been updated to the latest version on the IIS server and is working correctly in Microsoft Edge. However, Google Chrome still displays a “Not Secure” warning even though the latest certificate is visible.