Comments| Lobsters
At the Migrate and Modernize Summit, we’re announcing a set of agentic AI solutions and offerings to speed transformation and reduce friction. Learn more.| Microsoft Azure Blog
interactive shell navigation and history| Codeberg.org
I am trying to understand how to properly store ordered information in a relational database. An example: Say I have a Playlist, consisting of Songs. Inside my Relational Database, I have a table...| Software Engineering Stack Exchange
Introduction Versatile as it is, SQLite still suffers from one major drawback. Which is write concurrency. SQLite, using the Write-Ahead-Log (WAL) journaling mode, supports an unlimited number of r…| Oldmoe's blog
My career in data started as a SQL Server performance specialist, which meant I was deep into the nuances of indexes, locking and blocking, execution plan analysis and query design. These days I’m more in the world of the open table format such as Apache Iceberg. Having learned the internals of both| Jack Vanlightly
Ratcheting with Postgres CONSTRAINT| andrewjudson.com
Comparing the Django ORM support across official database backends, so you don’t have to learn it the hard way.| Paolo Melchiorre
🎨| temp.chrismorgan.info
Building a Hybrid Data Lakehouse with SQL Server PolyBase, FlashArray, and FlashBlade: Part 1 by Pure Storage Blog This article provides a step-by-step guide for integrating Microsoft SQL Server PolyBase with Pure Storage FlashArray and FlashBlade to create a powerful and flexible hybrid data lakehouse. The post Building a Hybrid Data Lakehouse with SQL Server PolyBase, FlashArray, and FlashBlade: Part 1 appeared first on Pure Storage Blog.| Pure Storage Blog
Born to Run: Unlocking the True Value of Oracle Data with Pure Storage by Pure Storage Blog We’ve been innovating and developing new paths for Oracle customers to take advantage of their storage platform to maximize their Oracle environment and unlock the full value of their data. Here’s what we’ve been working on. The post Born to Run: Unlocking the True Value of Oracle Data with Pure Storage appeared first on Pure Storage Blog.| Pure Storage Blog
Bytecode vs. Machine Code by Pure Storage Blog In this article, we look at both machine code and bytecode to unpack how they differ, what they both do, and how they’re related. The post Bytecode vs. Machine Code appeared first on Pure Storage Blog.| Pure Storage Blog
Get AI-Ready: Modernize Your Analytics with Pure Storage and Starburst by Pure Storage Blog Explore a new joint solution from Pure Storage and Starburst that helps organizations build a modern data architecture that can support tomorrow’s analytics and AI demands. The post Get AI-Ready: Modernize Your Analytics with Pure Storage and Starburst appeared first on Pure Storage Blog.| Pure Storage Blog
OSWALD is a Write-Ahead Log (WAL) design built exclusively on object storage primitives. It works with any object storage service that…| nvartolomei.com
Налаштування моніторингу кластеру AWS OpenSearch Service з CloudWatch, та моніторинг пам'яті JVM та k-NN, створення Grafana dashboard та алертів в Alertmanager| RTFM: Linux, DevOps та системне адміністрування | De...
We’re pleased to introduce the pgvector extension on Heroku Postgres . In an era where large language models (LLMs) and AI applications are paramount, pgvector provides the essential capability for performing high-dimensional vector similarity searches. This allows Heroku Postgres to quickly find similar data points in complex data, which is great for applications like recommendation systems and prompt engineering for LLMs. As of today, pgvector is fully compatible with all Production-tier ...| Heroku
Understanding query planner quirks yielded a ~35% speedup.| Evan Schwartz
In the first part, we covered the basics of AWS OpenSearch Service in general and the types of instances for Data Nodes – AWS: Getting Started with OpenSearch Service as a Vector Store. In the second part, we covered access, AWS: Creating an OpenSearch Service Cluster and Configuring Authentication and Authorization. Now let’s write… Read More » The post Terraform: creating an AWS OpenSearch Service cluster and users first appeared on RTFM: Linux, DevOps, and system administration.| RTFM: Linux, DevOps, and system administration
We are currently using AWS OpenSearch Service as a vector store for our RAG with AWS Bedrock Knowledge Base. We will talk more about RAG and Bedrock another time, but today let’s take a look at AWS OpenSearch Service. The task is to migrate our AWS OpenSearch Service Serverless to Managed, primarily due to… Read More » The post AWS: introduction to the OpenSearch Service as a vector store first appeared on RTFM: Linux, DevOps, and system administration.| RTFM: Linux, DevOps, and system administration
In the previous part, AWS: Getting Started with OpenSearch Service as a Vector Store, we looked at AWS OpenSearch Service in general, figured out how data is organized in it, what shards and nodes are, and what types of instances we actually need for data nodes. The next step is to create a cluster… Read More » The post AWS: creating an OpenSearch Service cluster and configuring authentication and authorization first appeared on RTFM: Linux, DevOps, and system administration.| RTFM: Linux, DevOps, and system administration
In the case of distributed, high-throughput string interning, horizontal scaling can be achieved by breaking up one large keyspace that requires strict coordination into billions of smaller keyspaces that can be randomly load-balanced across.| Jaz’s Blog
There are books & many articles online, like this one arguing for using Postgres for everything. I thought I’d take a look at one use case - using Postgres instead of Redis for caching. I work with APIs quite a bit, so I’d build a super simple HTTP server that responds with data from that cache. I’d start from Redis as this is something I frequently encounter at work, switch it out to Postgres using unlogged tables and see if there’s a difference.| Dizzy zone
The [PostgreSQL Global Development Group](https://www.postgresql.org) today announced the release of [PostgreSQL 18](https://www.postgresql.org/docs/18/release-18.html), the latest version of the world's most advanced …| PostgreSQL News
One of the significant improvements in PG18 is AIO. What are some basic tuning recommendations?| Tomas Vondra
Happy September! School is back in session for most, and fall is just around the corner. To kick off the school year, the Idaho Commission for Libraries is conducting a comprehensive 12-month campaign to highlight the diverse array of databases available at no cost to all Idahoans through LiLI.org. But we can't do it| Idaho Commission for Libraries
Discover how we reengineered our metrics storage engine for massive scale with Rust, a shard-per-core model, and real-time performance.| Datadog
В першій частині розібрались з основами AWS OpenSearch Service взагалі, і з типами інстансів для Data Nodes – AWS: знайомство з OpenSearch Service в ролі vector store. В другій – з доступами, AWS: створення OpenSearch Service cluster та налаштування аутентифікації і авторизації. Тепер напишемо Terraform code для створення ...| RTFM: Linux, DevOps та системне адміністрування
Build PostgreSQL wire-compatible databases in Python, powered by Rust. Serve data from Pandas, DuckDB, Polars over PostgreSQL protocol.| riffq.com
Effortless data collection and analysis| blog.incrementalforgetting.tech
Software Engineer | Open Source Enthusiast| hp77-creator.github.io
After a user signaled a performance issue in a Matrix client, we have added new tracing timers in the Matrix Rust SDK to spot the problem. Once found, we have fixed an SQL query improving the throughput from 19k to 251k events/sec, and the speed from 502ms to 39ms. Then after another creative patch, the throughput has been improved to 4.2M events/sec, and the speed to 2ms.| mnt.io
All the core components of pgEdge Distributed Postgres, along with any other pgEdge repositories that previously used the pgEdge Community License have now been re-licenced under the permissive PostgreSQL License, as approved by the Open Source Initiative!| www.pgedge.com
CedarDB is a database system that delivers unmatched performance for transactions and analytics, from small writes to handling billions of rows. Built on cutting-edge research to power today’s tools and tomorrow’s challenges.| cedardb.com
Searchlight Cyber's Security Research team details a Novel Technique for SQL Injection in PDO's Prepared Statements.| Searchlight Cyber
Missing the Forest for the Sequence Trees.| lewiscampbell.tech
Learn how queues make horizontal scaling, scheduling, and flow control easier in cloud systems, and how to make them durable and observable.| www.dbos.dev
The oldest recorded transaction was in 3100 BC| avi.im
DuckDB v1.3.0 significantly improved the scalability of geospatial joins with a dedicated SPATIAL_JOIN operator.| DuckDB
Why do we use caches at all? Can databases fully replace them?| avi.im
В попередній частині – AWS: знайомство з OpenSearch Service в ролі vector store – подивились на AWS OpenSearch Service взагалі, трохи розібрались з тим, як в ньому організовані дані, що таке shards та nodes, і які нам власне типи інстансів для data nodes треба. Наступний крок – створити кластер і подивитись н...| RTFM: Linux, DevOps та системне адміністрування
Знайомство з AWS OpenSearch Service в ролі vector store для AWS Bedrock Knowledge Base та планування ресурсів OpenSearch кластеру| RTFM: Linux, DevOps та системне адміністрування | De...
Database migration is one of those tasks that can either go smoothly or turn into a nightmare depending on your preparation. If you're considering migrating from MySQL to PostgreSQL, you're making a smart choice – PostgreSQL offers superior data integrity, better JSON support, advanced indexing, and robust ACID compliance. However, the migration process requires careful planning and understanding of the differences between these two database systems. In this comprehensive guide, I'll walk y...| TechPlanet
Is SQLite durable by default? What settings guarantee durability? The documentation and even comments from its creator give conflicting answers.| www.agwa.name
---| www.canicula.com
The WITH Clause| sqlite.org
Object storage is the primary storage solution for OLAP databases. This survey highlights major database players that have embraced this movement.| MinIO Blog
And why I don't want my database to choose the best encoding for me (yet)| Discover the Performance Engineer in you. | Polar Signals
I spent some time before making Datalogs that translated into SQL. https://www.philipzucker.com/tiny-sqlite-datalog/| Hey There Buddo!
SQLite when used with WAL doesn’t do fsync unless specified.| avi.im
🚨TL;DR: If you are a SurrealDB user running any SurrealDB instance backed by the RocksDB or SurrealKV storage backends you MUST EXPLICITLY set SURREAL_SYNC_DATA=true in your environment variables otherwise your instance is NOT crash safe and can very easily corrupt. If you're familiar with Rust or| ChillFish8's Blog
As programmers we spend a lot of time shuttling data back and forth between different systems and transforming it from one format to another. Sometimes it gets pretty miserable!| sophiebits.com
Funny and weird behavior with modern SSD devices, and how it impacts query execution.| Tomas Vondra
L2AW| law-theorem.com
ISRAELI INTELLIGENCE IS USING Microsoft’s cloud service to store recordings and metadata from millions of intercepted telephone conversations placed by residents of Gaza and the West Bank, according to a new investigation.| intelNews.org
Dynamo, DynamoDB, and Aurora DSQL| brooker.co.za
$$ $$ Category theory often sheds light on old problems by redescribing them in a conceptually cleaner way, but it less frequently gets used to develop concrete algorithms for practical problems. In this post, the problem we address involves a query we care about: we want to maintain the answer set to some query (e.g. “how many paths of length two are there in this graph?”) when the thing being queried is changing frequently. If the changes are frequent enough, we don’t want to have to...| Topos Institute
Today, we’re excited to announce the private preview of the Postgres Change Data Capture (CDC) connector in ClickPipes! This enables customers to replicate their Postgres databases to ClickHouse Cloud in just a few clicks and leverage ClickHouse for ...| PeerDB Blog
Last month, we acquired PeerDB, a company that specializes in Postgres CDC. PeerDB makes it fast and simple to replicate data from Postgres to ClickHouse. A common question from PeerDB users is how to model their data in ClickHouse after the replicat...| PeerDB Blog
Providing a fast and simple way to replicate data from Postgres to ClickHouse has been a top priority for us over the past few months. Last month, we acquired PeerDB, a company that specializes in Postgres CDC. We're actively integrating PeerDB into ...| PeerDB Blog
We are thrilled to join forces with ClickHouse to make it seamless for customers to move data from their Postgres databases to ClickHouse and power real-time analytics and data warehousing use cases. We released the ClickHouse target connector for Po...| PeerDB Blog
At PeerDB, security has always been a top priority. Our customers trust us with their critical data, and we are dedicated to upholding the highest standards of data protection and security. We are excited to announce that PeerDB has achieved SOC 2 Ty...| PeerDB Blog
At PeerDB, we are building a fast and simple way to replicate data from Postgres to data warehouses like Snowflake, ClickHouse etc. and queues such as Kafka, Redpanda etc. We implement Postgres Change Data Capture (CDC) to reliably replicate changes ...| PeerDB Blog
Today, PeerDB is pleased to announce that our target connector for Elasticsearch is now in beta. Elasticsearch is a popular search engine system underpinned by a distributed document database, and we have been seeing a lot of use cases for Elasticsea...| PeerDB Blog
🚀 Today, we're excited to announce that PeerDB Cloud is officially entering public beta. If you're a data engineer or an organization looking for a fast, simple, and cost-effective way to replicate data from Postgres to data warehouses such as Snowf...| PeerDB Blog
pg_dump and pg_restore are reliable tools for backing up and restoring Postgres databases. They're essential for database migrations, disaster recovery and so on. They offer precise control over object selection for backup/restore, dump format option...| PeerDB Blog
PeerDB offers a fast and cost-effective way to move data from PostgreSQL to data warehouses, such as Snowflake, and to queues like Kafka. This enables businesses to have real-time and reliable access to data, which is of utmost importance in this AI ...| PeerDB Blog
Introduction Logical Replication is one of the many ways a Postgres database can replicate data to other Postgres database (a.k.a standby). Logical replication directly reads from the write-ahead log (WAL), recording every database change, avoiding t...| PeerDB Blog
At PeerDB, we are building a fast and a cost-effective way to replicate data from Postgres to Data Warehouses such as Snowflake, BigQuery, ClickHouse, Postgres and so on. All our customers run Postgres at the heart of the data stack, running fully ma...| PeerDB Blog
Inspired by the 1BR Challenge, I wanted to see how much it would cost to transfer 1 billion rows from Postgres to Snowflake. Moving 1 billion rows is no easy task. The process involves not just the transfer of data but ensuring its integrity, error r...| PeerDB Blog
This paper from SIGMOD 2016 proposes a transaction healing approach to improve the scalability of Optimistic Concurrency Control (OCC) in m...| muratbuffalo.blogspot.com
Zero-ETL search and analytics for Postgres| ParadeDB
Why build auth logic when your database can do it better? Learn how SQL Server user impersonation creates stronger security boundaries with less application complexity.| Alonso Network
Eliminating PostgreSQL catalog traversal overhead with local partition calculations for up to 20x faster hash partition queries.| Shayon Mukherjee
The data lake was once heralded as the future, an infinitely scalable reservoir for all our raw data, promising to transform it into actionable insights. This was a logical progression from databases and data warehouses, each step driven by the increasing demand for scalability. Yet, in embracing the data lake's| MinIO Blog
Frequent UC Davis users may have noticed in June 2025 that the database has a new look resulting from a migration to the new platform. This new upgraded environment is intended to improve the literature search experience. Users still have access to key features, including access to their search history, CINAHL subject headings, and reliable set of filters.| UC Davis Library
1 Background| jepsen.io
Contributed by SpiralDB, Vortex is an extensible, next-generation columnar storage format designed for building high-performance, future-proof data systems| www.linuxfoundation.org
GT2 Pro members, download a high-res version of this image that you can use royalty-free anywhere:| Good Tech Things
A tale of frustrated dreams, utopian user experiences and calculated tradeoffs.| paoramen
This EuroSys '23 paper reads like an SOSP best paper. Maybe it helped that EuroSys 2023 was in Rome. Academic conferences are more enjoyabl...| muratbuffalo.blogspot.com
NP-Incompleteness:| www.kuniga.me
I've been thinking lately about this piece from Frank Lantz about the Thielian "the Olympics, but you take performance-enhancing drugs." The pitch being...| buttondown.com
SQLite WAL has checksums, but on corruption it drops all the data and does not raise error| avi.im
This is a beginner’s guide to hacking into Turso DB (formerly known as Limbo), the SQLite rewrite in Rust. I will explore how to get familiar with Turso’s codebase, tooling and tests| avi.im
Discover how a pull‑based, demand‑driven pipeline maximizes throughput with back-pressure controls.| Sequin blog
Common Pitfalls and Potential Optimizations in Django| hakibenita.com
On trying to mash up SQLite with ideas stolen from Accountants, Clojure, Datomic, XTDB, Rama, and Local-first-ers, to satisfy Henderson's Tenth Law. Viz., to make a sufficiently complicated data system containing an ad-hoc, informally-specified, bug-ridden, slow implementation of half of a bitemporal database. Because? Because laying about on a hammock, contemplating hopelessly complected objects like Current Databases isn't just for the Rich man.| www.evalapply.org
A story about a bug with logical replication slot creation on Postgres read replicas| ClickHouse
The Lower Saxony State Criminal Police Office has analysed digital evidence from the alleged RAF militant Daniela Klette using artificial intelligence. This disruptive technology is reminiscent of BKA pioneer Horst Herold. In the 1970s, the then-President of the German Federal Criminal Police Office (BKA), Horst Herold, introduced electronic data processing. Since then, Germany has maintained […] Der Beitrag Commissioner Computer vs. Daniela Klette: Use of AI in RAF investigations raises qu...| Matthias Monroy
Before PostgreSQL 10, Postgres users partitioned their data using inheritance based partitioning. The method used constraints to define the partitions and rules or triggers to route the data to appropriate partition. A user had to write and maintain code for all that. PostgreSQL 10 introduced declarative partitioning, which is much easier to setup and requires almost no maintenance. PostgreSQL 11| Walking with the Elephants
PostgreSQL 10 is full with a lot of big, new and exciting features. Declarative partitioning is one of those. It is something users have wanted for years. During PGCon 2017, it was a hot topic of discussion. People wanted to know more about the feature, and were eager to try it out. The un-conference and conference session on partitioning attracted a large crowd. One of the frequently asked questions centred on whether a user can change partition bounds of an existing partition. This bears ex...| Walking with the Elephants
When implementing an optimization for derived clause lookup myself, Amit Langote and David Rowley argued about the initial size of hash tabl...| ashutoshpg.blogspot.com
I've always been told that caching is a tool to make software faster. That, given some careful considerations to consistency, caching makes it so that when...| buttondown.com
An evaluation of how often Postgres picks the optimal plan for simple scan queries.| Tomas Vondra
A simple insert query turned into a silent performance killer. Our frontend pings our server every few minutes to track device activity. Each ping attempts to insert a row into a DevicePingDaily table, which has a unique constraint on (device_id, date) to ensure only one record per device per day. In Django, the logic looked like this: try: DevicePingDaily.objects.create(device=device, date=today) except IntegrityError: pass It seemed harmless. But as traffic grew, latency spiked and API time...| Johnny Metz
Lessons from building a parser| Log Blog Kebab
Today we'll explain how do modern databases allow us to perform backups without blocking - thus enabling users to operate on them while the backup is beign made continuously in the background. We'll also show how this approach allow us to restore database to any point| Bartosz Sypytkowski
