Join us

ContentUpdates and recent posts about BigQuery..
Link
@faun shared a link, 5 months, 2 weeks ago
FAUN.dev()

Containers: Everything You Need To Know

cgroupsand namespaces anchor Linux containers, isolating resources and processes like gatekeepers with a mission. On macOS and Windows, these containers ride in VMs withWSL2orLinuxKit, putting on their "welcome to the virtual world" hats. EnterrunC, executing OCI-built images with isolation flair, w.. read more  

Containers: Everything You Need To Know
Link
@faun shared a link, 5 months, 2 weeks ago
FAUN.dev()

Crawling a billion web pages in just over 24 hours

Imagine tearing through1 billion pages in a single dayon a shoestring budget. This crawler pulled it off with12 nodes and some savvy async maneuvering. But here's the kicker: it wasn’t the fetching that choked the CPU. Nope, it was the parsing. Today’s web behemoths, bloated with JavaScript and othe.. read more  

Crawling a billion web pages in just over 24 hours
Link
@faun shared a link, 5 months, 2 weeks ago
FAUN.dev()

Understand CPU Branch Instructions Better

Branch prediction matters. Why? About a quarter of instructions are branches, and modern CPUs nail an accuracyabove 90%. Yet, those often-pesky branches can choke CPUs, stalling instruction flow. So, take a wrench to yourif-else logic. Trim indirect branches whenever you can—your CPU will thank you... read more  

Link
@faun shared a link, 5 months, 2 weeks ago
FAUN.dev()

Lessons from scaling PostgreSQL queues to 100K events

PostgreSQLjuggles 100,000 events per second. Just needs some index wizardry and query twerking. The problem? Table bloat and Write Amplification. Gross. Enter the mightyCOPY—it bulldozes through bulk data, politely ignoring the usualInsertdrag. And those recursiveCTEs? They pull off loose index scan.. read more  

Lessons from scaling PostgreSQL queues to 100K events
Link
@faun shared a link, 5 months, 2 weeks ago
FAUN.dev()

How Go 1.24's Swiss Tables saved us hundreds of gigabytes

Uncovered a memory regression inGo 1.24. Pored over memory patterns in countless pods like a detective with too much caffeine. Pinpointed sneaky allocation blunders... read more  

Link
@faun shared a link, 5 months, 2 weeks ago
FAUN.dev()

Parsing 1 Billion Rows in Bun/Typescript Under 10 Seconds

Buntries to swallow files over 4GB and promptly chokes. The culprit? ItsBuffercaps out at 4GB. The fix? Slice files into chunks under 4GB but keep the buffer lean, no more than 128KB, to keep things zippy. Pull out the big guns—workers. This move fires up all CPU cores, slashing processing time from.. read more  

Parsing 1 Billion Rows in Bun/Typescript Under 10 Seconds
Link
@faun shared a link, 5 months, 2 weeks ago
FAUN.dev()

Death by a thousand slops

By 2025,AI slopwill infect20%of curl's security submissions. Meanwhile, a mere5%reveal actual threats. Cutting the$90,000bounty might fend off the slopsters, but it'll scare away the real wizards, too... read more  

Death by a thousand slops
Link
@faun shared a link, 5 months, 2 weeks ago
FAUN.dev()

The Micro-Frontend Architecture Handbook

iframes: Secure and isolated, but clunky as dial-up. Best for legacy cleanup missions.Web Components: Native and framework-agnostic, perfect for reusable UI with Shadow DOM flair.single-spa: Juggles multiple SPAs with the finesse of a circus, though it gets chatty.Module Federation: Real-time module.. read more  

The Micro-Frontend Architecture Handbook
Link
@faun shared a link, 5 months, 2 weeks ago
FAUN.dev()

AV1 @ Scale: Film Grain Synthesis, The Awakening

AV1 Film Grain Synthesis (FGS)tricks the eye by imitating film grain after compression. Cuts bitrates like a ninja and keeps the artistry alive. Models grasp grain's pattern and punch, ensuring sharp visuals on bandwidth-challenged gadgets. Grainy magic, delivered neatly!.. read more  

AV1 @ Scale: Film Grain Synthesis, The Awakening
Link
@faun shared a link, 5 months, 2 weeks ago
FAUN.dev()

Scalability is not performance

Boostingscalabilityin distributed systems isn't just a mad dash for speed. It's about morphing resources to tackle shifting demand. Nail scalability, and you balance infrastructure costs with job handling efficiency, all while juggling resource utilization at a sweet spot around 0.5. Crave a drama-f.. read more  

Scalability is not performance
BigQuery is a cloud-native, serverless analytics platform designed to store, query, and analyze massive volumes of structured and semi-structured data using standard SQL. It separates storage from compute, automatically scales resources, and eliminates the need for infrastructure management, indexing, or capacity planning.

BigQuery is optimized for analytical workloads such as business intelligence, log analysis, data science, and machine learning. It supports real-time data ingestion via streaming, batch loading from cloud storage, and federated queries across external data sources like Cloud Storage, Bigtable, and Google Drive.

Query execution is distributed and highly parallel, enabling interactive performance even on petabyte-scale datasets. The platform integrates deeply with the Google Cloud ecosystem, including Looker for BI, Vertex AI for ML workflows, Dataflow for streaming pipelines, and BigQuery ML, which allows users to train and run machine learning models directly using SQL.

Built-in security features include fine-grained IAM controls, column- and row-level security, encryption by default, and audit logging. BigQuery follows a consumption-based pricing model, charging for storage and queries (on-demand or reserved capacity).