r/programminghorror • u/dex4er • 3d ago
GitHub's HTTP header
This is the crazy thing I've found after analysis why the CI test fails with "Maximum header size exceeded" today.
437
u/Belialson 3d ago
Company I work at was proxing some endpoints to us, it was a big shop. They used to set a cookie with json containing entire user’s session including every page visited and it’s details.
Same effect but unfortunately on production
81
u/buckypimpin 3d ago
isnt a cookie max 4KiB?
107
u/RegisteredJustToSay 3d ago
That's per cookie (name + value). You can store many times that if you break it up over multiple keys, tho max 180 per host on chrome. You could fit ten text copies of Alice in wonderland in that space.
You're right though, of course. Just don't want to give anyone here the idea that you're limited to this little space per domain.
30
u/RustOnTheEdge 3d ago
Alice in Wonderland is roughly 170kb, so it would fit 4.3 times in all the available header space.
15
11
u/RegisteredJustToSay 2d ago
Weird, specifically checked gutenberg and saw it was under 80 kB but now on desktop it says 170 kB. Was on mobile though so may have been a presentation issue. Thanks for the correction!
7
24
u/caboosetp 3d ago
You can fit quite a bit in 4k before it causes issues that a browser restart can fix and push off the issue.
5
u/Chesterlespaul 3d ago
Interesting idea. Obviously don’t do this. But I wonder sort of data this could be useful for.
12
u/groumly 3d ago
Asp (can’t remember if it was the og or the .net one, I think .net) did a lot of crap like that back in day.
It was used to turn stateless web framework into a stateful mode iirc.
9
u/lenswipe 3d ago
Ah yes __VIEWSTATE
I remember seeing that hidden field in a form, decoding it and was like ".....no fucking way"
5
u/groumly 3d ago
Oh, that rings a bell, yes. Wait, so was it actually stored in cookies then? Or did I get confused and it was just passed in a hidden form on every damn link of every damn page?
7
u/lenswipe 2d ago
no it was a hidden form field that got posted between pages
3
1
u/Chesterlespaul 3d ago
Oh that would be so useful back in the day! Reactive state set from web requests.
120
u/n9iels 3d ago
Proof that managing CSP headers is an absolute nightmare
-24
u/AyrA_ch 3d ago
They could have gone with
*.blob.core.windows.netto massively shorten the header.63
u/Alikont 3d ago
That's a generic domain for all Azure Blob storage accounts. So I could register my own Storage Account and get the matching url.
26
u/AyrA_ch 3d ago
They also allow
raw.githubusercontent.comwhich serves user defined files, so serving unspecified 3rd party content is not the primary concern here to not use an asterisk.3
u/Joniator 3d ago
Cant be sure, maybe they only query well known paths, then raw.githubusercontent.com would be fine, some random guys blob who controls that storagr wouldnt.
But thats grasping at straws, i dont think they had concern for anything but that was the fix for some issue they didn't understand enough to come up with a reasonable solution.
4
u/reerden 3d ago
Which makes it even dumber as it’s trivial to add your own CNAME to storage accounts.
2
u/RustOnTheEdge 3d ago
Which came with significant downsides back in the day. Only http or you’d need to use azure front door or something, iirc
1
u/reerden 2d ago
This is still the case, but I don’t think GitHub has any issue affording front door or another content delivery service.
If you run into limitations of a single storage account that you need multiple, especially this many, then surely you can afford something like cloudflare or front door.
0
u/Online_Matter 3d ago
Should they have used a reverse proxy then?
2
u/n9iels 1d ago
In terms of managing CSP headers that would be easier in theory. In praktische however, most analytics libraries have statics urls and marketing collegues don't care and will still create tickets to "whitelist absolutelyrandom.whatever.example.com".
1
u/Online_Matter 1d ago
That's what I was thinking and you're painfully right. Lot of urls in the post seems to be for the same project however.
37
u/steeltownsquirrel 3d ago
I'm off to write the first hit song in the blob.core genre
1
14
9
u/TheSpiffySpaceman 2d ago
standard Microsoft fare, unfortunately.
I dare you to write an Azure DO extension and retain your sanity. I promise you that it's impossible.
9
u/hursofid 1d ago
Yeah. I've been troubleshooting why Teams voice/video freezes on Linux and have discovered that it downloads a fucking 200-something megabytes of cookies IIRC.
Fuck microslop.
-10
u/des_the_furry 1d ago
Weak ahh OS can’t handle 200 mb 😹
6
2
6
3
3
u/andyfsu99 1d ago
Can someone ELI5 why this is this way?
3
u/GuyWithTwoThumbs 1d ago
Apparently too hard to write a proxy service to forward requests to relevant services, where you'd have to re-authenticate the user to each service as authentication cookies for each domain would not be passed by the browser to this proxy service. This is just how CSP works, and its an important security measure to have. Easier to have the client authenticate with the relevant cookies, etc. directly to the relevant services.
1
u/Hurtfulbirch 1d ago
A CSP tells the browser what 3rd party domains are allowed through. For things like fonts, scripts, images, etc. It’s to prevent xss attacks mostly, I think.
4
u/GoddammitDontShootMe [ $[ $RANDOM % 6 ] == 0 ] && rm -rf / || echo “You live” 3d ago
Christ. And that isn't even the full list in the screenshot.
1
1
u/ryanmgarber 8h ago
GitHub becoming one of the buggiest, most annoying services to exist was not something I ever expected to happen and yet Microslop found a fucking way
1
u/topgamer7 6h ago
I had to try to set up CSP for my work. It's a body nightmare if any of your js is third party. You have to call out every domain that code is loaded from at all
54
u/Julius_Babies 3d ago edited 18h ago
I saw this post a while ago and just ran into this. The HTTP Engine my app uses cannot handle header values longer than 8192 characters by default, which causes my in-app update mechanism (which relies on GitHub releases) to fail when trying to download the update. I cannot provide a fix for this as this fix would require - you guessed it - an update.
This has been reported as a bug on the GitHub Community discussions already: https://github.com/orgs/community/discussions/211142
Edit: It's now fixed