ContentPosts from @denyskontorskyy..
Link
@faun shared a link, 2 weeks, 6 days ago

Crawling a billion web pages in just over 24 hours

Imagine tearing through1 billion pages in a single dayon a shoestring budget. This crawler pulled it off with12 nodes and some savvy async maneuvering. But here's the kicker: it wasn’t the fetching that choked the CPU. Nope, it was the parsing. Today’s web behemoths, bloated with JavaScript and othe..

Crawling a billion web pages in just over 24 hours
Link
@faun shared a link, 2 weeks, 6 days ago

Containers: Everything You Need To Know

cgroupsand namespaces anchor Linux containers, isolating resources and processes like gatekeepers with a mission. On macOS and Windows, these containers ride in VMs withWSL2orLinuxKit, putting on their "welcome to the virtual world" hats. EnterrunC, executing OCI-built images with isolation flair, w..

Containers: Everything You Need To Know
Link
@faun shared a link, 2 weeks, 6 days ago

The Micro-Frontend Architecture Handbook

iframes: Secure and isolated, but clunky as dial-up. Best for legacy cleanup missions.Web Components: Native and framework-agnostic, perfect for reusable UI with Shadow DOM flair.single-spa: Juggles multiple SPAs with the finesse of a circus, though it gets chatty.Module Federation: Real-time module..

The Micro-Frontend Architecture Handbook
Link
@faun shared a link, 2 weeks, 6 days ago

Parsing 1 Billion Rows in Bun/Typescript Under 10 Seconds

Buntries to swallow files over 4GB and promptly chokes. The culprit? ItsBuffercaps out at 4GB. The fix? Slice files into chunks under 4GB but keep the buffer lean, no more than 128KB, to keep things zippy. Pull out the big guns—workers. This move fires up all CPU cores, slashing processing time from..

Parsing 1 Billion Rows in Bun/Typescript Under 10 Seconds
Link
@faun shared a link, 2 weeks, 6 days ago

How Go 1.24's Swiss Tables saved us hundreds of gigabytes

Uncovered a memory regression inGo 1.24. Pored over memory patterns in countless pods like a detective with too much caffeine. Pinpointed sneaky allocation blunders...

Link
@faun shared a link, 2 weeks, 6 days ago

Death by a thousand slops

By 2025,AI slopwill infect20%of curl's security submissions. Meanwhile, a mere5%reveal actual threats. Cutting the$90,000bounty might fend off the slopsters, but it'll scare away the real wizards, too...

Death by a thousand slops
Link
@faun shared a link, 2 weeks, 6 days ago

Scalability is not performance

Boostingscalabilityin distributed systems isn't just a mad dash for speed. It's about morphing resources to tackle shifting demand. Nail scalability, and you balance infrastructure costs with job handling efficiency, all while juggling resource utilization at a sweet spot around 0.5. Crave a drama-f..

Scalability is not performance
Link
@faun shared a link, 2 weeks, 6 days ago

AV1 @ Scale: Film Grain Synthesis, The Awakening

AV1 Film Grain Synthesis (FGS)tricks the eye by imitating film grain after compression. Cuts bitrates like a ninja and keeps the artistry alive. Models grasp grain's pattern and punch, ensuring sharp visuals on bandwidth-challenged gadgets. Grainy magic, delivered neatly!..

AV1 @ Scale: Film Grain Synthesis, The Awakening
Link
@faun shared a link, 2 weeks, 6 days ago

Lessons from scaling PostgreSQL queues to 100K events

PostgreSQLjuggles 100,000 events per second. Just needs some index wizardry and query twerking. The problem? Table bloat and Write Amplification. Gross. Enter the mightyCOPY—it bulldozes through bulk data, politely ignoring the usualInsertdrag. And those recursiveCTEs? They pull off loose index scan..

Lessons from scaling PostgreSQL queues to 100K events
Link
@faun shared a link, 2 weeks, 6 days ago

Rethinking CLI interfaces for AI

LLMs fumble with CLI tools because they lack context. Tweaking APIs and tools for LLM savvy could cut mistakes and boost context efficiency.Smarter interfaces might keep them from getting stuck in infinite loops or bungling directories, slashing tool calls and making automation crisp and tidy...

Rethinking CLI interfaces for AI