際際滷shows by User: supercoco9 / http://www.slideshare.net/images/logo.gif 際際滷shows by User: supercoco9 / Fri, 04 Oct 2024 15:47:58 GMT 際際滷Share feed for 際際滷shows by User: supercoco9 The Future of Fast Databases: Lessons from a Decade of QuestDB /slideshow/the-future-of-fast-databases-lessons-from-a-decade-of-questdb/272192275 futureoffastdbsbigdatalondon-241004154758-f94fea4d
Over the last decade, QuestDB has been at the forefront of handling time series data with a focus on speed and efficiency. In this talk, Ill share practical insights from our experience serving thousands of users, highlighting what weve learned about building and maintaining a fast database that can ingest millions of events per second. QuestDB, an open-source time series database, has traditionally relied on a custom-built, non-standard data storage format designed for performance. As we move forward, were actively developing its architecture to support open formats like Apache Parquet and Arrow, reflecting a broader industry shift. Ill discuss the engineering challenges weve faced during this transition, the new possibilities it creates, and why these changes are crucial for the evolving database landscape. Through live demos, Ill showcase QuestDBs performance in real-time data ingestion and queries, and demonstrate some of the features enabled by these new formats.]]>
Over the last decade, QuestDB has been at the forefront of handling time series data with a focus on speed and efficiency. In this talk, Ill share practical insights from our experience serving thousands of users, highlighting what weve learned about building and maintaining a fast database that can ingest millions of events per second. QuestDB, an open-source time series database, has traditionally relied on a custom-built, non-standard data storage format designed for performance. As we move forward, were actively developing its architecture to support open formats like Apache Parquet and Arrow, reflecting a broader industry shift. Ill discuss the engineering challenges weve faced during this transition, the new possibilities it creates, and why these changes are crucial for the evolving database landscape. Through live demos, Ill showcase QuestDBs performance in real-time data ingestion and queries, and demonstrate some of the features enabled by these new formats.]]> Fri, 04 Oct 2024 15:47:58 GMT /slideshow/the-future-of-fast-databases-lessons-from-a-decade-of-questdb/272192275 supercoco9@slideshare.net(supercoco9) The Future of Fast Databases: Lessons from a Decade of QuestDB supercoco9 Over the last decade, QuestDB has been at the forefront of handling time series data with a focus on speed and efficiency. In this talk, Ill share practical insights from our experience serving thousands of users, highlighting what weve learned about building and maintaining a fast database that can ingest millions of events per second. QuestDB, an open-source time series database, has traditionally relied on a custom-built, non-standard data storage format designed for performance. As we move forward, were actively developing its architecture to support open formats like Apache Parquet and Arrow, reflecting a broader industry shift. Ill discuss the engineering challenges weve faced during this transition, the new possibilities it creates, and why these changes are crucial for the evolving database landscape. Through live demos, Ill showcase QuestDBs performance in real-time data ingestion and queries, and demonstrate some of the features enabled by these new formats. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/futureoffastdbsbigdatalondon-241004154758-f94fea4d-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Over the last decade, QuestDB has been at the forefront of handling time series data with a focus on speed and efficiency. In this talk, Ill share practical insights from our experience serving thousands of users, highlighting what weve learned about building and maintaining a fast database that can ingest millions of events per second. QuestDB, an open-source time series database, has traditionally relied on a custom-built, non-standard data storage format designed for performance. As we move forward, were actively developing its architecture to support open formats like Apache Parquet and Arrow, reflecting a broader industry shift. Ill discuss the engineering challenges weve faced during this transition, the new possibilities it creates, and why these changes are crucial for the evolving database landscape. Through live demos, Ill showcase QuestDBs performance in real-time data ingestion and queries, and demonstrate some of the features enabled by these new formats.
The Future of Fast Databases: Lessons from a Decade of QuestDB from javier ramirez
]]> 89 0 https://cdn.slidesharecdn.com/ss_thumbnails/futureoffastdbsbigdatalondon-241004154758-f94fea4d-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 C坦mo hemos implementado sem叩ntica de "Exactly Once" en nuestra base de datos de alto rendimiento /slideshow/como-hemos-implementado-semantica-de-exactly-once-en-nuestra-base-de-datos-de-alto-rendimiento/270065157 questexactlyoncecommitconf-240704152526-ea886719
Los sistemas distribuidos son dif鱈ciles. Los sistemas distribuidos de alto rendimiento, m叩s. Latencias de red, mensajes sin confirmaci坦n de recibo, reinicios de servidores, fallos de hardware, bugs en el software, releases problem叩ticas, timeouts... hay un mont坦n de motivos por los que es muy dif鱈cil saber si un mensaje que has enviado se ha recibido y procesado correctamente en destino. As鱈 que para asegurar mandas el mensaje otra vez.. y otra... y cruzas los dedos para que el sistema del otro lado tenga tolerancia a los duplicados. QuestDB es una base de datos open source dise単ada para alto rendimiento. Nos quer鱈amos asegurar de poder ofrecer garant鱈as de "exactly once", deduplicando mensajes en tiempo de ingesti坦n. En esta charla, te cuento c坦mo dise単amos e implementamos la palabra clave DEDUP en QuestDB, permitiendo deduplicar y adem叩s permitiendo Upserts en datos en tiempo real, a単adiendo solo un 8% de tiempo de proceso, incluso en flujos con millones de inserciones por segundo. Adem叩s, explicar辿 nuestra arquitectura de log de escrituras (WAL) paralelo y multithread. Por supuesto, todo esto te lo cuento con demos, para que veas c坦mo funciona en la pr叩ctica.]]>
Los sistemas distribuidos son dif鱈ciles. Los sistemas distribuidos de alto rendimiento, m叩s. Latencias de red, mensajes sin confirmaci坦n de recibo, reinicios de servidores, fallos de hardware, bugs en el software, releases problem叩ticas, timeouts... hay un mont坦n de motivos por los que es muy dif鱈cil saber si un mensaje que has enviado se ha recibido y procesado correctamente en destino. As鱈 que para asegurar mandas el mensaje otra vez.. y otra... y cruzas los dedos para que el sistema del otro lado tenga tolerancia a los duplicados. QuestDB es una base de datos open source dise単ada para alto rendimiento. Nos quer鱈amos asegurar de poder ofrecer garant鱈as de "exactly once", deduplicando mensajes en tiempo de ingesti坦n. En esta charla, te cuento c坦mo dise単amos e implementamos la palabra clave DEDUP en QuestDB, permitiendo deduplicar y adem叩s permitiendo Upserts en datos en tiempo real, a単adiendo solo un 8% de tiempo de proceso, incluso en flujos con millones de inserciones por segundo. Adem叩s, explicar辿 nuestra arquitectura de log de escrituras (WAL) paralelo y multithread. Por supuesto, todo esto te lo cuento con demos, para que veas c坦mo funciona en la pr叩ctica.]]> Thu, 04 Jul 2024 15:25:26 GMT /slideshow/como-hemos-implementado-semantica-de-exactly-once-en-nuestra-base-de-datos-de-alto-rendimiento/270065157 supercoco9@slideshare.net(supercoco9) C坦mo hemos implementado sem叩ntica de "Exactly Once" en nuestra base de datos de alto rendimiento supercoco9 Los sistemas distribuidos son dif鱈ciles. Los sistemas distribuidos de alto rendimiento, m叩s. Latencias de red, mensajes sin confirmaci坦n de recibo, reinicios de servidores, fallos de hardware, bugs en el software, releases problem叩ticas, timeouts... hay un mont坦n de motivos por los que es muy dif鱈cil saber si un mensaje que has enviado se ha recibido y procesado correctamente en destino. As鱈 que para asegurar mandas el mensaje otra vez.. y otra... y cruzas los dedos para que el sistema del otro lado tenga tolerancia a los duplicados. QuestDB es una base de datos open source dise単ada para alto rendimiento. Nos quer鱈amos asegurar de poder ofrecer garant鱈as de "exactly once", deduplicando mensajes en tiempo de ingesti坦n. En esta charla, te cuento c坦mo dise単amos e implementamos la palabra clave DEDUP en QuestDB, permitiendo deduplicar y adem叩s permitiendo Upserts en datos en tiempo real, a単adiendo solo un 8% de tiempo de proceso, incluso en flujos con millones de inserciones por segundo. Adem叩s, explicar辿 nuestra arquitectura de log de escrituras (WAL) paralelo y multithread. Por supuesto, todo esto te lo cuento con demos, para que veas c坦mo funciona en la pr叩ctica. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/questexactlyoncecommitconf-240704152526-ea886719-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Los sistemas distribuidos son dif鱈ciles. Los sistemas distribuidos de alto rendimiento, m叩s. Latencias de red, mensajes sin confirmaci坦n de recibo, reinicios de servidores, fallos de hardware, bugs en el software, releases problem叩ticas, timeouts... hay un mont坦n de motivos por los que es muy dif鱈cil saber si un mensaje que has enviado se ha recibido y procesado correctamente en destino. As鱈 que para asegurar mandas el mensaje otra vez.. y otra... y cruzas los dedos para que el sistema del otro lado tenga tolerancia a los duplicados. QuestDB es una base de datos open source dise単ada para alto rendimiento. Nos quer鱈amos asegurar de poder ofrecer garant鱈as de &quot;exactly once&quot;, deduplicando mensajes en tiempo de ingesti坦n. En esta charla, te cuento c坦mo dise単amos e implementamos la palabra clave DEDUP en QuestDB, permitiendo deduplicar y adem叩s permitiendo Upserts en datos en tiempo real, a単adiendo solo un 8% de tiempo de proceso, incluso en flujos con millones de inserciones por segundo. Adem叩s, explicar辿 nuestra arquitectura de log de escrituras (WAL) paralelo y multithread. Por supuesto, todo esto te lo cuento con demos, para que veas c坦mo funciona en la pr叩ctica.
Cmo hemos implementado semntica de "Exactly Once" en nuestra base de datos de alto rendimiento from javier ramirez
]]> 49 0 https://cdn.slidesharecdn.com/ss_thumbnails/questexactlyoncecommitconf-240704152526-ea886719-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 How We Added Replication to QuestDB - JonTheBeach /slideshow/how-we-added-replication-to-questdb-jonthebeach/270065108 howweaddedreplicationtoquestdb-240704152142-3a888164
Building a database that can beat industry benchmarks is hard work, and we had to use every trick in the book to keep as close to the hardware as possible. In doing so, we initially decided QuestDB would scale only vertically, on a single instance. A few years later, data replication for horizontally scaling reads and for high availability became one of the most demanded features, especially for enterprise and cloud environments. So, we rolled up our sleeves and made it happen. Today, QuestDB supports an unbounded number of geographically distributed read-replicas without slowing down reads on the primary node, which can ingest data at over 4 million rows per second. In this talk, I will tell you about the technical decisions we made, and their trade offs. You'll learn how we had to revamp the whole ingestion layer, and how we actually made the primary faster than before when we added multi-threaded Write Ahead Logs to deal with data replication. I'll also discuss how we are leveraging object storage as a central part of the process. And of course, I'll show you a live demo of high-performance multi-region replication in action.]]>
Building a database that can beat industry benchmarks is hard work, and we had to use every trick in the book to keep as close to the hardware as possible. In doing so, we initially decided QuestDB would scale only vertically, on a single instance. A few years later, data replication for horizontally scaling reads and for high availability became one of the most demanded features, especially for enterprise and cloud environments. So, we rolled up our sleeves and made it happen. Today, QuestDB supports an unbounded number of geographically distributed read-replicas without slowing down reads on the primary node, which can ingest data at over 4 million rows per second. In this talk, I will tell you about the technical decisions we made, and their trade offs. You'll learn how we had to revamp the whole ingestion layer, and how we actually made the primary faster than before when we added multi-threaded Write Ahead Logs to deal with data replication. I'll also discuss how we are leveraging object storage as a central part of the process. And of course, I'll show you a live demo of high-performance multi-region replication in action.]]> Thu, 04 Jul 2024 15:21:41 GMT /slideshow/how-we-added-replication-to-questdb-jonthebeach/270065108 supercoco9@slideshare.net(supercoco9) How We Added Replication to QuestDB - JonTheBeach supercoco9 Building a database that can beat industry benchmarks is hard work, and we had to use every trick in the book to keep as close to the hardware as possible. In doing so, we initially decided QuestDB would scale only vertically, on a single instance. A few years later, data replication for horizontally scaling reads and for high availability became one of the most demanded features, especially for enterprise and cloud environments. So, we rolled up our sleeves and made it happen. Today, QuestDB supports an unbounded number of geographically distributed read-replicas without slowing down reads on the primary node, which can ingest data at over 4 million rows per second. In this talk, I will tell you about the technical decisions we made, and their trade offs. You'll learn how we had to revamp the whole ingestion layer, and how we actually made the primary faster than before when we added multi-threaded Write Ahead Logs to deal with data replication. I'll also discuss how we are leveraging object storage as a central part of the process. And of course, I'll show you a live demo of high-performance multi-region replication in action. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/howweaddedreplicationtoquestdb-240704152142-3a888164-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Building a database that can beat industry benchmarks is hard work, and we had to use every trick in the book to keep as close to the hardware as possible. In doing so, we initially decided QuestDB would scale only vertically, on a single instance. A few years later, data replication for horizontally scaling reads and for high availability became one of the most demanded features, especially for enterprise and cloud environments. So, we rolled up our sleeves and made it happen. Today, QuestDB supports an unbounded number of geographically distributed read-replicas without slowing down reads on the primary node, which can ingest data at over 4 million rows per second. In this talk, I will tell you about the technical decisions we made, and their trade offs. You&#39;ll learn how we had to revamp the whole ingestion layer, and how we actually made the primary faster than before when we added multi-threaded Write Ahead Logs to deal with data replication. I&#39;ll also discuss how we are leveraging object storage as a central part of the process. And of course, I&#39;ll show you a live demo of high-performance multi-region replication in action.
How We Added Replication to QuestDB - JonTheBeach from javier ramirez
]]> 199 0 https://cdn.slidesharecdn.com/ss_thumbnails/howweaddedreplicationtoquestdb-240704152142-3a888164-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 The Building Blocks of QuestDB, a Time Series Database /slideshow/the-building-blocks-of-a-time-series-database/269538848 vlccodesthebuildingblocksoftimeseriesdatabase-240606084916-3b508aa8
Talk Delivered at Valencia Codes Meetup 2024-06. Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds. It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.]]>
Talk Delivered at Valencia Codes Meetup 2024-06. Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds. It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.]]> Thu, 06 Jun 2024 08:49:16 GMT /slideshow/the-building-blocks-of-a-time-series-database/269538848 supercoco9@slideshare.net(supercoco9) The Building Blocks of QuestDB, a Time Series Database supercoco9 Talk Delivered at Valencia Codes Meetup 2024-06. Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds. It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/vlccodesthebuildingblocksoftimeseriesdatabase-240606084916-3b508aa8-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Talk Delivered at Valencia Codes Meetup 2024-06. Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds. It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
The Building Blocks of QuestDB, a Time Series Database from javier ramirez
]]> 125 0 https://cdn.slidesharecdn.com/ss_thumbnails/vlccodesthebuildingblocksoftimeseriesdatabase-240606084916-3b508aa8-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 多Se puede vivir del open source? T3chfest https://es.slideshare.net/slideshow/se-puede-vivir-del-open-source-t3chfest/266806457 canyoumakealivingoutofopensource-240315153150-11d939d0
Hubo un tiempo en el que casi cualquier componente de software requer鱈a pagar una licencia. Afortunadamente, hoy en d鱈a gracias al software libre y de c坦digo abierto, se puede desarrollar pr叩cticamente cualquier aplicaci坦n usando componentes gratuitos. Pero, si el software es gratis, 多Qui辿n lo desarrolla? 多Trabaja la comunidad de software libre de forma altruista? 多Se puede desarrollar software libre de forma profesional? De hecho, hay quien dice que el c坦digo abierto tal y como lo conocimos ya no existe, y que lo que hay hoy en d鱈a es otra cosa. En esta charla hablar辿 de c坦mo se puede monetizar el c坦digo libre, y de algunos posibles conflictos que puedes encontrarte en el camino. Adem叩s, te contar辿 c坦mo hacemos desde QuestDB para desarrollar una base de datos de c坦digo abierto y mantener un equipo estable viviendo de ello. Comentar辿 tambi辿n algunas situaciones problem叩ticas a las que proyectos muy destacados se han enfrentado, o que se enfrentan a d鱈a de hoy.]]>
Hubo un tiempo en el que casi cualquier componente de software requer鱈a pagar una licencia. Afortunadamente, hoy en d鱈a gracias al software libre y de c坦digo abierto, se puede desarrollar pr叩cticamente cualquier aplicaci坦n usando componentes gratuitos. Pero, si el software es gratis, 多Qui辿n lo desarrolla? 多Trabaja la comunidad de software libre de forma altruista? 多Se puede desarrollar software libre de forma profesional? De hecho, hay quien dice que el c坦digo abierto tal y como lo conocimos ya no existe, y que lo que hay hoy en d鱈a es otra cosa. En esta charla hablar辿 de c坦mo se puede monetizar el c坦digo libre, y de algunos posibles conflictos que puedes encontrarte en el camino. Adem叩s, te contar辿 c坦mo hacemos desde QuestDB para desarrollar una base de datos de c坦digo abierto y mantener un equipo estable viviendo de ello. Comentar辿 tambi辿n algunas situaciones problem叩ticas a las que proyectos muy destacados se han enfrentado, o que se enfrentan a d鱈a de hoy.]]> Fri, 15 Mar 2024 15:31:50 GMT https://es.slideshare.net/slideshow/se-puede-vivir-del-open-source-t3chfest/266806457 supercoco9@slideshare.net(supercoco9) 多Se puede vivir del open source? T3chfest supercoco9 Hubo un tiempo en el que casi cualquier componente de software requer鱈a pagar una licencia. Afortunadamente, hoy en d鱈a gracias al software libre y de c坦digo abierto, se puede desarrollar pr叩cticamente cualquier aplicaci坦n usando componentes gratuitos. Pero, si el software es gratis, 多Qui辿n lo desarrolla? 多Trabaja la comunidad de software libre de forma altruista? 多Se puede desarrollar software libre de forma profesional? De hecho, hay quien dice que el c坦digo abierto tal y como lo conocimos ya no existe, y que lo que hay hoy en d鱈a es otra cosa. En esta charla hablar辿 de c坦mo se puede monetizar el c坦digo libre, y de algunos posibles conflictos que puedes encontrarte en el camino. Adem叩s, te contar辿 c坦mo hacemos desde QuestDB para desarrollar una base de datos de c坦digo abierto y mantener un equipo estable viviendo de ello. Comentar辿 tambi辿n algunas situaciones problem叩ticas a las que proyectos muy destacados se han enfrentado, o que se enfrentan a d鱈a de hoy. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/canyoumakealivingoutofopensource-240315153150-11d939d0-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Hubo un tiempo en el que casi cualquier componente de software requer鱈a pagar una licencia. Afortunadamente, hoy en d鱈a gracias al software libre y de c坦digo abierto, se puede desarrollar pr叩cticamente cualquier aplicaci坦n usando componentes gratuitos. Pero, si el software es gratis, 多Qui辿n lo desarrolla? 多Trabaja la comunidad de software libre de forma altruista? 多Se puede desarrollar software libre de forma profesional? De hecho, hay quien dice que el c坦digo abierto tal y como lo conocimos ya no existe, y que lo que hay hoy en d鱈a es otra cosa. En esta charla hablar辿 de c坦mo se puede monetizar el c坦digo libre, y de algunos posibles conflictos que puedes encontrarte en el camino. Adem叩s, te contar辿 c坦mo hacemos desde QuestDB para desarrollar una base de datos de c坦digo abierto y mantener un equipo estable viviendo de ello. Comentar辿 tambi辿n algunas situaciones problem叩ticas a las que proyectos muy destacados se han enfrentado, o que se enfrentan a d鱈a de hoy.
from javier ramirez
]]> 39 0 https://cdn.slidesharecdn.com/ss_thumbnails/canyoumakealivingoutofopensource-240315153150-11d939d0-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 QuestDB: The building blocks of a fast open-source time-series database /slideshow/questdb-the-building-blocks-of-a-fast-opensource-timeseries-database/264639767 questdbthebuildingblocksoftimeseriesdatabaseosacon-231214105903-8d30b550
(talk delivered at OSA CON 23) Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds. It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will learn how it deals with data ingestion, and which SQL extensions it implements for working with time-series efficiently. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or data deduplication.]]>
(talk delivered at OSA CON 23) Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds. It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will learn how it deals with data ingestion, and which SQL extensions it implements for working with time-series efficiently. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or data deduplication.]]> Thu, 14 Dec 2023 10:59:03 GMT /slideshow/questdb-the-building-blocks-of-a-fast-opensource-timeseries-database/264639767 supercoco9@slideshare.net(supercoco9) QuestDB: The building blocks of a fast open-source time-series database supercoco9 (talk delivered at OSA CON 23) Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds. It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will learn how it deals with data ingestion, and which SQL extensions it implements for working with time-series efficiently. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or data deduplication. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/questdbthebuildingblocksoftimeseriesdatabaseosacon-231214105903-8d30b550-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> (talk delivered at OSA CON 23) Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds. It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will learn how it deals with data ingestion, and which SQL extensions it implements for working with time-series efficiently. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or data deduplication.
QuestDB: The building blocks of a fast open-source time-series database from javier ramirez
]]> 219 0 https://cdn.slidesharecdn.com/ss_thumbnails/questdbthebuildingblocksoftimeseriesdatabaseosacon-231214105903-8d30b550-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB, la base de datos de time series open source /slideshow/como-creamos-questdb-cloud-un-saas-basado-en-kubernetes-alrededor-de-questdb-la-base-de-datos-de-time-series-open-source-259139942/259139942 base-de-datos-open-source-saas-kubernetes-230711155342-daa633d1
QuestDB es una base de datos open source de alto rendimiento. Mucha gente nos comentaba que les gustar鱈a usarla como servicio, sin tener que gestionar las m叩quinas. As鱈 que nos pusimos manos a la obra para desarrollar una soluci坦n que nos permitiese lanzar instancias de QuestDB con provisionado, monitorizaci坦n, seguridad o actualizaciones totalmente gestionadas. Unos cuantos clusters de Kubernetes m叩s tarde, conseguimos lanzar nuestra oferta de QuestDB Cloud. Esta charla es la historia de c坦mo llegamos ah鱈. Hablar辿 de herramientas como Calico, Karpenter, CoreDNS, Telegraf, Prometheus, Loki o Grafana, pero tambi辿n de retos como autenticaci坦n, facturaci坦n, multi-nube, o de a qu辿 tienes que decir que no para poder sobrevivir en la nube.]]>
QuestDB es una base de datos open source de alto rendimiento. Mucha gente nos comentaba que les gustar鱈a usarla como servicio, sin tener que gestionar las m叩quinas. As鱈 que nos pusimos manos a la obra para desarrollar una soluci坦n que nos permitiese lanzar instancias de QuestDB con provisionado, monitorizaci坦n, seguridad o actualizaciones totalmente gestionadas. Unos cuantos clusters de Kubernetes m叩s tarde, conseguimos lanzar nuestra oferta de QuestDB Cloud. Esta charla es la historia de c坦mo llegamos ah鱈. Hablar辿 de herramientas como Calico, Karpenter, CoreDNS, Telegraf, Prometheus, Loki o Grafana, pero tambi辿n de retos como autenticaci坦n, facturaci坦n, multi-nube, o de a qu辿 tienes que decir que no para poder sobrevivir en la nube.]]> Tue, 11 Jul 2023 15:53:41 GMT /slideshow/como-creamos-questdb-cloud-un-saas-basado-en-kubernetes-alrededor-de-questdb-la-base-de-datos-de-time-series-open-source-259139942/259139942 supercoco9@slideshare.net(supercoco9) Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB, la base de datos de time series open source supercoco9 QuestDB es una base de datos open source de alto rendimiento. Mucha gente nos comentaba que les gustar鱈a usarla como servicio, sin tener que gestionar las m叩quinas. As鱈 que nos pusimos manos a la obra para desarrollar una soluci坦n que nos permitiese lanzar instancias de QuestDB con provisionado, monitorizaci坦n, seguridad o actualizaciones totalmente gestionadas. Unos cuantos clusters de Kubernetes m叩s tarde, conseguimos lanzar nuestra oferta de QuestDB Cloud. Esta charla es la historia de c坦mo llegamos ah鱈. Hablar辿 de herramientas como Calico, Karpenter, CoreDNS, Telegraf, Prometheus, Loki o Grafana, pero tambi辿n de retos como autenticaci坦n, facturaci坦n, multi-nube, o de a qu辿 tienes que decir que no para poder sobrevivir en la nube. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/base-de-datos-open-source-saas-kubernetes-230711155342-daa633d1-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> QuestDB es una base de datos open source de alto rendimiento. Mucha gente nos comentaba que les gustar鱈a usarla como servicio, sin tener que gestionar las m叩quinas. As鱈 que nos pusimos manos a la obra para desarrollar una soluci坦n que nos permitiese lanzar instancias de QuestDB con provisionado, monitorizaci坦n, seguridad o actualizaciones totalmente gestionadas. Unos cuantos clusters de Kubernetes m叩s tarde, conseguimos lanzar nuestra oferta de QuestDB Cloud. Esta charla es la historia de c坦mo llegamos ah鱈. Hablar辿 de herramientas como Calico, Karpenter, CoreDNS, Telegraf, Prometheus, Loki o Grafana, pero tambi辿n de retos como autenticaci坦n, facturaci坦n, multi-nube, o de a qu辿 tienes que decir que no para poder sobrevivir en la nube.
Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB, la base de datos de time series open source from javier ramirez
]]> 75 0 https://cdn.slidesharecdn.com/ss_thumbnails/base-de-datos-open-source-saas-kubernetes-230711155342-daa633d1-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database - Berlin Buzzwords /slideshow/ingesting-over-four-million-rows-per-second-with-questdb-timeseries-database-berlin-buzzwords/258511776 berlin-buzzwords-ingesting-over-four-million-rows-per-second-230620083119-09013eb7
How would you build a database to support sustained ingestion of several hundreds of thousands rows per second while running near real-time queries on top? In this session I will go over some of the technical decisions and trade-offs we applied when building QuestDB, an open source time-series database developed mainly in JAVA, and how we can achieve over four million row writes per second on a single instance without blocking or slowing down the reads. There will be code and demos, of course. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.]]>
How would you build a database to support sustained ingestion of several hundreds of thousands rows per second while running near real-time queries on top? In this session I will go over some of the technical decisions and trade-offs we applied when building QuestDB, an open source time-series database developed mainly in JAVA, and how we can achieve over four million row writes per second on a single instance without blocking or slowing down the reads. There will be code and demos, of course. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.]]> Tue, 20 Jun 2023 08:31:19 GMT /slideshow/ingesting-over-four-million-rows-per-second-with-questdb-timeseries-database-berlin-buzzwords/258511776 supercoco9@slideshare.net(supercoco9) Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database - Berlin Buzzwords supercoco9 How would you build a database to support sustained ingestion of several hundreds of thousands rows per second while running near real-time queries on top? In this session I will go over some of the technical decisions and trade-offs we applied when building QuestDB, an open source time-series database developed mainly in JAVA, and how we can achieve over four million row writes per second on a single instance without blocking or slowing down the reads. There will be code and demos, of course. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/berlin-buzzwords-ingesting-over-four-million-rows-per-second-230620083119-09013eb7-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> How would you build a database to support sustained ingestion of several hundreds of thousands rows per second while running near real-time queries on top? In this session I will go over some of the technical decisions and trade-offs we applied when building QuestDB, an open source time-series database developed mainly in JAVA, and how we can achieve over four million row writes per second on a single instance without blocking or slowing down the reads. There will be code and demos, of course. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database - Berlin Buzzwords from javier ramirez
]]> 98 0 https://cdn.slidesharecdn.com/ss_thumbnails/berlin-buzzwords-ingesting-over-four-million-rows-per-second-230620083119-09013eb7-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Deduplicating and analysing time-series data with Apache Beam and QuestDB /slideshow/deduplicating-and-analysing-timeseries-data-with-apache-beam-and-questdb/258486628 deduplicatingandanalysingtime-seriesdatawithapachebeamandquestdb-230619004945-20218bd9
Time series data pipelines tend to prioritise speed and freshness over completeness and integrity. In such scenarios, it is very common to ingest duplicate data, which may be fine for many analytical use cases, but is very inconvenient for others. There are many open source databases built specifically for the speed and query semantics of time series, and most of them lack automatic deduplication of events in near real-time. One such database is QuestDB, which requires a manual batch process to deduplicate ingested data. In this talk, we will see how we can successfully use Apache Beam to deduplicate streaming time series, which can then be analysed by a time series database.]]>
Time series data pipelines tend to prioritise speed and freshness over completeness and integrity. In such scenarios, it is very common to ingest duplicate data, which may be fine for many analytical use cases, but is very inconvenient for others. There are many open source databases built specifically for the speed and query semantics of time series, and most of them lack automatic deduplication of events in near real-time. One such database is QuestDB, which requires a manual batch process to deduplicate ingested data. In this talk, we will see how we can successfully use Apache Beam to deduplicate streaming time series, which can then be analysed by a time series database.]]> Mon, 19 Jun 2023 00:49:45 GMT /slideshow/deduplicating-and-analysing-timeseries-data-with-apache-beam-and-questdb/258486628 supercoco9@slideshare.net(supercoco9) Deduplicating and analysing time-series data with Apache Beam and QuestDB supercoco9 Time series data pipelines tend to prioritise speed and freshness over completeness and integrity. In such scenarios, it is very common to ingest duplicate data, which may be fine for many analytical use cases, but is very inconvenient for others. There are many open source databases built specifically for the speed and query semantics of time series, and most of them lack automatic deduplication of events in near real-time. One such database is QuestDB, which requires a manual batch process to deduplicate ingested data. In this talk, we will see how we can successfully use Apache Beam to deduplicate streaming time series, which can then be analysed by a time series database. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/deduplicatingandanalysingtime-seriesdatawithapachebeamandquestdb-230619004945-20218bd9-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Time series data pipelines tend to prioritise speed and freshness over completeness and integrity. In such scenarios, it is very common to ingest duplicate data, which may be fine for many analytical use cases, but is very inconvenient for others. There are many open source databases built specifically for the speed and query semantics of time series, and most of them lack automatic deduplication of events in near real-time. One such database is QuestDB, which requires a manual batch process to deduplicate ingested data. In this talk, we will see how we can successfully use Apache Beam to deduplicate streaming time series, which can then be analysed by a time series database.
Deduplicating and analysing time-series data with Apache Beam and QuestDB from javier ramirez
]]> 52 0 https://cdn.slidesharecdn.com/ss_thumbnails/deduplicatingandanalysingtime-seriesdatawithapachebeamandquestdb-230619004945-20218bd9-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Your Database Cannot Do this (well) /slideshow/your-database-cannot-do-this-well/258129608 ramirezjavierwedyourdatabasecannotdothiswell-230530115707-c0999e78
Relational databases were created a long time ago for a simpler world. Even if they are still awesome tools for generic workloads, there are some things they cannot do well. In this session I will speak about purpose-built databases that you can use for specific business scenarios. We will see the type of queries you can run on a Graph database, a Document Database, and a Time-Series database. We will then see how a relational database could also be used for the same use cases, just in a much more complex way.]]>
Relational databases were created a long time ago for a simpler world. Even if they are still awesome tools for generic workloads, there are some things they cannot do well. In this session I will speak about purpose-built databases that you can use for specific business scenarios. We will see the type of queries you can run on a Graph database, a Document Database, and a Time-Series database. We will then see how a relational database could also be used for the same use cases, just in a much more complex way.]]> Tue, 30 May 2023 11:57:07 GMT /slideshow/your-database-cannot-do-this-well/258129608 supercoco9@slideshare.net(supercoco9) Your Database Cannot Do this (well) supercoco9 Relational databases were created a long time ago for a simpler world. Even if they are still awesome tools for generic workloads, there are some things they cannot do well. In this session I will speak about purpose-built databases that you can use for specific business scenarios. We will see the type of queries you can run on a Graph database, a Document Database, and a Time-Series database. We will then see how a relational database could also be used for the same use cases, just in a much more complex way. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/ramirezjavierwedyourdatabasecannotdothiswell-230530115707-c0999e78-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Relational databases were created a long time ago for a simpler world. Even if they are still awesome tools for generic workloads, there are some things they cannot do well. In this session I will speak about purpose-built databases that you can use for specific business scenarios. We will see the type of queries you can run on a Graph database, a Document Database, and a Time-Series database. We will then see how a relational database could also be used for the same use cases, just in a much more complex way.
Your Database Cannot Do this (well) from javier ramirez
]]> 21 0 https://cdn.slidesharecdn.com/ss_thumbnails/ramirezjavierwedyourdatabasecannotdothiswell-230530115707-c0999e78-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Your Timestamps Deserve Better than a Generic Database /slideshow/your-timestamps-deserve-better-than-a-generic-database/258014633 your-timestamps-deserve-better-230525082819-04b9201f
If you are storing records with a timestamp in your database, it is very likely a time series database can make your life easier. However, time series databases are still the great unknown for a large part of the tech community. In this talk, I will show you what use cases they are good for, what they give you that you cannot get from a traditional database, and when it is a good idea (and when it is not) to use them. For the demos, we will be using QuestDB, the fastest open-source time series database. ]]>
If you are storing records with a timestamp in your database, it is very likely a time series database can make your life easier. However, time series databases are still the great unknown for a large part of the tech community. In this talk, I will show you what use cases they are good for, what they give you that you cannot get from a traditional database, and when it is a good idea (and when it is not) to use them. For the demos, we will be using QuestDB, the fastest open-source time series database. ]]> Thu, 25 May 2023 08:28:19 GMT /slideshow/your-timestamps-deserve-better-than-a-generic-database/258014633 supercoco9@slideshare.net(supercoco9) Your Timestamps Deserve Better than a Generic Database supercoco9 If you are storing records with a timestamp in your database, it is very likely a time series database can make your life easier. However, time series databases are still the great unknown for a large part of the tech community. In this talk, I will show you what use cases they are good for, what they give you that you cannot get from a traditional database, and when it is a good idea (and when it is not) to use them. For the demos, we will be using QuestDB, the fastest open-source time series database. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/your-timestamps-deserve-better-230525082819-04b9201f-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> If you are storing records with a timestamp in your database, it is very likely a time series database can make your life easier. However, time series databases are still the great unknown for a large part of the tech community. In this talk, I will show you what use cases they are good for, what they give you that you cannot get from a traditional database, and when it is a good idea (and when it is not) to use them. For the demos, we will be using QuestDB, the fastest open-source time series database.
Your Timestamps Deserve Better than a Generic Database from javier ramirez
]]> 26 0 https://cdn.slidesharecdn.com/ss_thumbnails/your-timestamps-deserve-better-230525082819-04b9201f-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 C坦mo se dise単a una base de datos que pueda ingerir m叩s de cuatro millones de eventos por segundo /slideshow/cmo-se-disea-una-base-de-datos-que-pueda-ingerir-ms-de-cuatro-millones-de-eventos-por-segundo/257503870 commit-como-se-disena-una-base-de-datos-4-millones-por-segundo-230421105531-ef5f9016
En esta sesi坦n voy a contar las decisiones t辿cnicas que tomamos al desarrollar QuestDB, una base de datos Open Source para series temporales compatible con Postgres, y c坦mo conseguimos escribir m叩s de cuatro millones de filas por segundo sin bloquear o enlentecer las consultas. Hablar辿 de cosas como (zero) Garbage Collection, vectorizaci坦n de instrucciones usando SIMD, reescribir en lugar de reutilizar para ara単ar microsegundos, aprovecharse de los avances en procesadores, discos duros y sistemas operativos, como por ejemplo el soporte de io_uring, o del balance entre experiencia de usuario y rendimiento cuando se plantean nuevas funcionalidades. ]]>
En esta sesi坦n voy a contar las decisiones t辿cnicas que tomamos al desarrollar QuestDB, una base de datos Open Source para series temporales compatible con Postgres, y c坦mo conseguimos escribir m叩s de cuatro millones de filas por segundo sin bloquear o enlentecer las consultas. Hablar辿 de cosas como (zero) Garbage Collection, vectorizaci坦n de instrucciones usando SIMD, reescribir en lugar de reutilizar para ara単ar microsegundos, aprovecharse de los avances en procesadores, discos duros y sistemas operativos, como por ejemplo el soporte de io_uring, o del balance entre experiencia de usuario y rendimiento cuando se plantean nuevas funcionalidades. ]]> Fri, 21 Apr 2023 10:55:31 GMT /slideshow/cmo-se-disea-una-base-de-datos-que-pueda-ingerir-ms-de-cuatro-millones-de-eventos-por-segundo/257503870 supercoco9@slideshare.net(supercoco9) C坦mo se dise単a una base de datos que pueda ingerir m叩s de cuatro millones de eventos por segundo supercoco9 En esta sesi坦n voy a contar las decisiones t辿cnicas que tomamos al desarrollar QuestDB, una base de datos Open Source para series temporales compatible con Postgres, y c坦mo conseguimos escribir m叩s de cuatro millones de filas por segundo sin bloquear o enlentecer las consultas. Hablar辿 de cosas como (zero) Garbage Collection, vectorizaci坦n de instrucciones usando SIMD, reescribir en lugar de reutilizar para ara単ar microsegundos, aprovecharse de los avances en procesadores, discos duros y sistemas operativos, como por ejemplo el soporte de io_uring, o del balance entre experiencia de usuario y rendimiento cuando se plantean nuevas funcionalidades. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/commit-como-se-disena-una-base-de-datos-4-millones-por-segundo-230421105531-ef5f9016-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> En esta sesi坦n voy a contar las decisiones t辿cnicas que tomamos al desarrollar QuestDB, una base de datos Open Source para series temporales compatible con Postgres, y c坦mo conseguimos escribir m叩s de cuatro millones de filas por segundo sin bloquear o enlentecer las consultas. Hablar辿 de cosas como (zero) Garbage Collection, vectorizaci坦n de instrucciones usando SIMD, reescribir en lugar de reutilizar para ara単ar microsegundos, aprovecharse de los avances en procesadores, discos duros y sistemas operativos, como por ejemplo el soporte de io_uring, o del balance entre experiencia de usuario y rendimiento cuando se plantean nuevas funcionalidades.
Cmo se disea una base de datos que pueda ingerir ms de cuatro millones de eventos por segundo from javier ramirez
]]> 58 0 https://cdn.slidesharecdn.com/ss_thumbnails/commit-como-se-disena-una-base-de-datos-4-millones-por-segundo-230421105531-ef5f9016-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 QuestDB-Community-Call-20220728 /slideshow/questdbcommunitycall20220728/253545258 questdb-community-call-20220728-221013113531-b8c67891
際際滷s for the QuestDB community call in July 2022 Video available at https://www.youtube.com/watch?v=PfjFT78jlfQ&t=2003s&ab_channel=QuestDB]]>
際際滷s for the QuestDB community call in July 2022 Video available at https://www.youtube.com/watch?v=PfjFT78jlfQ&t=2003s&ab_channel=QuestDB]]> Thu, 13 Oct 2022 11:35:30 GMT /slideshow/questdbcommunitycall20220728/253545258 supercoco9@slideshare.net(supercoco9) QuestDB-Community-Call-20220728 supercoco9 際際滷s for the QuestDB community call in July 2022 Video available at https://www.youtube.com/watch?v=PfjFT78jlfQ&t=2003s&ab_channel=QuestDB <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/questdb-community-call-20220728-221013113531-b8c67891-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> 際際滷s for the QuestDB community call in July 2022 Video available at https://www.youtube.com/watch?v=PfjFT78jlfQ&amp;t=2003s&amp;ab_channel=QuestDB
QuestDB-Community-Call-20220728 from javier ramirez
]]> 24 0 https://cdn.slidesharecdn.com/ss_thumbnails/questdb-community-call-20220728-221013113531-b8c67891-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Processing and analysing streaming data with Python. Pycon Italy 2022 /slideshow/processing-and-analysing-streaming-data-with-python-pycon-italy-2022/253545178 processingandanalysingstreamingdatawithpython-221013113101-4ad26d8d
Data used to be a batch thing, but more and more we get unbounded streams of data, fast or slow, that we need to process and analyse in near real time. In this talk Ill show you how you can use Apache Flink and QuestDB to build reliable streaming data pipelines that can grow as much as you need.]]>
Data used to be a batch thing, but more and more we get unbounded streams of data, fast or slow, that we need to process and analyse in near real time. In this talk Ill show you how you can use Apache Flink and QuestDB to build reliable streaming data pipelines that can grow as much as you need.]]> Thu, 13 Oct 2022 11:31:01 GMT /slideshow/processing-and-analysing-streaming-data-with-python-pycon-italy-2022/253545178 supercoco9@slideshare.net(supercoco9) Processing and analysing streaming data with Python. Pycon Italy 2022 supercoco9 Data used to be a batch thing, but more and more we get unbounded streams of data, fast or slow, that we need to process and analyse in near real time. In this talk Ill show you how you can use Apache Flink and QuestDB to build reliable streaming data pipelines that can grow as much as you need. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/processingandanalysingstreamingdatawithpython-221013113101-4ad26d8d-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Data used to be a batch thing, but more and more we get unbounded streams of data, fast or slow, that we need to process and analyse in near real time. In this talk Ill show you how you can use Apache Flink and QuestDB to build reliable streaming data pipelines that can grow as much as you need.
Processing and analysing streaming data with Python. Pycon Italy 2022 from javier ramirez
]]> 12 0 https://cdn.slidesharecdn.com/ss_thumbnails/processingandanalysingstreamingdatawithpython-221013113101-4ad26d8d-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 QuestDB: ingesting a million time series per second on a single instance. Big data london 2022.pdf /supercoco9/questdb-ingesting-a-million-time-series-per-second-on-a-single-instance-big-data-london-2022pdf ingesting1millionrowspersecond-221013112626-a3ee5fff
In this session I will show you the technical decisions we made when building QuestDB, the open source, Postgres compatible, time-series database, and how we can achieve a million row writes per second without blocking or slowing down the reads.]]>
In this session I will show you the technical decisions we made when building QuestDB, the open source, Postgres compatible, time-series database, and how we can achieve a million row writes per second without blocking or slowing down the reads.]]> Thu, 13 Oct 2022 11:26:26 GMT /supercoco9/questdb-ingesting-a-million-time-series-per-second-on-a-single-instance-big-data-london-2022pdf supercoco9@slideshare.net(supercoco9) QuestDB: ingesting a million time series per second on a single instance. Big data london 2022.pdf supercoco9 In this session I will show you the technical decisions we made when building QuestDB, the open source, Postgres compatible, time-series database, and how we can achieve a million row writes per second without blocking or slowing down the reads. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/ingesting1millionrowspersecond-221013112626-a3ee5fff-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> In this session I will show you the technical decisions we made when building QuestDB, the open source, Postgres compatible, time-series database, and how we can achieve a million row writes per second without blocking or slowing down the reads.
QuestDB: ingesting a million time series per second on a single instance. Big data london 2022.pdf from javier ramirez
]]> 312 0 https://cdn.slidesharecdn.com/ss_thumbnails/ingesting1millionrowspersecond-221013112626-a3ee5fff-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Servicios e infraestructura de AWS y la pr坦xima regi坦n en Arag坦n https://es.slideshare.net/slideshow/servicios-e-infraestructura-de-aws-y-la-prxima-regin-en-aragn/250856430 servicioseinfraestructuradeawsylaproximaregionenaragoncompressed-211217094717
AWS est叩 montando una regi坦n de infraestructura en Arag坦n. Vale, pero 多Qu辿 significa eso? 多Es tan diferente de un centro de datos convencional o de otros proveedores de nube? (Spoiler: S鱈). En esta sesi坦n te cuento por qu辿. Hay video en https://catedrasamcadt.unizar.es/noticias/el-momento-tecnologico-actual-contado-por-trabajadores-de-amazon-web-services/]]>
AWS est叩 montando una regi坦n de infraestructura en Arag坦n. Vale, pero 多Qu辿 significa eso? 多Es tan diferente de un centro de datos convencional o de otros proveedores de nube? (Spoiler: S鱈). En esta sesi坦n te cuento por qu辿. Hay video en https://catedrasamcadt.unizar.es/noticias/el-momento-tecnologico-actual-contado-por-trabajadores-de-amazon-web-services/]]> Fri, 17 Dec 2021 09:47:17 GMT https://es.slideshare.net/slideshow/servicios-e-infraestructura-de-aws-y-la-prxima-regin-en-aragn/250856430 supercoco9@slideshare.net(supercoco9) Servicios e infraestructura de AWS y la pr坦xima regi坦n en Arag坦n supercoco9 AWS est叩 montando una regi坦n de infraestructura en Arag坦n. Vale, pero 多Qu辿 significa eso? 多Es tan diferente de un centro de datos convencional o de otros proveedores de nube? (Spoiler: S鱈). En esta sesi坦n te cuento por qu辿. Hay video en https://catedrasamcadt.unizar.es/noticias/el-momento-tecnologico-actual-contado-por-trabajadores-de-amazon-web-services/ <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/servicioseinfraestructuradeawsylaproximaregionenaragoncompressed-211217094717-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> AWS est叩 montando una regi坦n de infraestructura en Arag坦n. Vale, pero 多Qu辿 significa eso? 多Es tan diferente de un centro de datos convencional o de otros proveedores de nube? (Spoiler: S鱈). En esta sesi坦n te cuento por qu辿. Hay video en https://catedrasamcadt.unizar.es/noticias/el-momento-tecnologico-actual-contado-por-trabajadores-de-amazon-web-services/
from javier ramirez
]]> 228 0 https://cdn.slidesharecdn.com/ss_thumbnails/servicioseinfraestructuradeawsylaproximaregionenaragoncompressed-211217094717-thumbnail.jpg?width=120&height=120&fit=bounds presentation 000000 http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Primeros pasos en desarrollo serverless /slideshow/primeros-pasos-en-desarrollo-serverless/249860160 primerospasosendesarrolloserverless-210726094809
多Qu辿 es eso del desarrollo sin servidores? 多Qu辿 lenguajes puedo utilizar? 多C坦mo hago cosas como autenticaci坦n, o guardar en base de datos, o enviar notificaciones? 多Esto escala? A todas estas preguntas, y a alguna m叩s, intentar辿 dar respuesta en esta sesi坦n, donde har辿 una peque単a demo de montar una app muy sencilla y desplegarla en la nube sin preocuparnos de gestionar infraestructura. Charla realizada por primera vez para AlcarriaConf 2021]]>
多Qu辿 es eso del desarrollo sin servidores? 多Qu辿 lenguajes puedo utilizar? 多C坦mo hago cosas como autenticaci坦n, o guardar en base de datos, o enviar notificaciones? 多Esto escala? A todas estas preguntas, y a alguna m叩s, intentar辿 dar respuesta en esta sesi坦n, donde har辿 una peque単a demo de montar una app muy sencilla y desplegarla en la nube sin preocuparnos de gestionar infraestructura. Charla realizada por primera vez para AlcarriaConf 2021]]> Mon, 26 Jul 2021 09:48:09 GMT /slideshow/primeros-pasos-en-desarrollo-serverless/249860160 supercoco9@slideshare.net(supercoco9) Primeros pasos en desarrollo serverless supercoco9 多Qu辿 es eso del desarrollo sin servidores? 多Qu辿 lenguajes puedo utilizar? 多C坦mo hago cosas como autenticaci坦n, o guardar en base de datos, o enviar notificaciones? 多Esto escala? A todas estas preguntas, y a alguna m叩s, intentar辿 dar respuesta en esta sesi坦n, donde har辿 una peque単a demo de montar una app muy sencilla y desplegarla en la nube sin preocuparnos de gestionar infraestructura. Charla realizada por primera vez para AlcarriaConf 2021 <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/primerospasosendesarrolloserverless-210726094809-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> 多Qu辿 es eso del desarrollo sin servidores? 多Qu辿 lenguajes puedo utilizar? 多C坦mo hago cosas como autenticaci坦n, o guardar en base de datos, o enviar notificaciones? 多Esto escala? A todas estas preguntas, y a alguna m叩s, intentar辿 dar respuesta en esta sesi坦n, donde har辿 una peque単a demo de montar una app muy sencilla y desplegarla en la nube sin preocuparnos de gestionar infraestructura. Charla realizada por primera vez para AlcarriaConf 2021
Primeros pasos en desarrollo serverless from javier ramirez
]]> 90 0 https://cdn.slidesharecdn.com/ss_thumbnails/primerospasosendesarrolloserverless-210726094809-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 How AWS is reinventing the cloud /supercoco9/how-aws-is-reinventing-the-cloud howawsisreinventingcloud-210624143919
AWS launched publicly on March 2006 with just one service, starting the age of the public cloud. You might think after 15 years everything in cloud has already been invented, but that's simply not the case. In this session I want to show you how AWS is reinventing the cloud in areas like computing, machine learning, databases and analytics, or cloud infrastructure.]]>
AWS launched publicly on March 2006 with just one service, starting the age of the public cloud. You might think after 15 years everything in cloud has already been invented, but that's simply not the case. In this session I want to show you how AWS is reinventing the cloud in areas like computing, machine learning, databases and analytics, or cloud infrastructure.]]> Thu, 24 Jun 2021 14:39:19 GMT /supercoco9/how-aws-is-reinventing-the-cloud supercoco9@slideshare.net(supercoco9) How AWS is reinventing the cloud supercoco9 AWS launched publicly on March 2006 with just one service, starting the age of the public cloud. You might think after 15 years everything in cloud has already been invented, but that's simply not the case. In this session I want to show you how AWS is reinventing the cloud in areas like computing, machine learning, databases and analytics, or cloud infrastructure. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/howawsisreinventingcloud-210624143919-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> AWS launched publicly on March 2006 with just one service, starting the age of the public cloud. You might think after 15 years everything in cloud has already been invented, but that&#39;s simply not the case. In this session I want to show you how AWS is reinventing the cloud in areas like computing, machine learning, databases and analytics, or cloud infrastructure.
How AWS is reinventing the cloud from javier ramirez
]]> 824 0 https://cdn.slidesharecdn.com/ss_thumbnails/howawsisreinventingcloud-210624143919-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Analitica de datos en tiempo real con Apache Flink y Apache BEAM /slideshow/analitica-de-datos-en-tiempo-real-con-apache-flink-y-apache-beam/239070440 analiticadedatosentiemporealapacheflinkapachebeam-201103163035
Trabajar en tiempo real con datos que se mueven muy r叩pido no es trivial, sobre todo con vol炭menes de datos elevados. Apache Flink y Apache BEAM est叩n espec鱈ficamente dise単adas para ese caso de uso. En esta charla te contar辿 los retos de la anal鱈tica en tiempo real, cu叩l es la arquitectura de Apache Flink, qu辿 es Apace BEAM, y c坦mo usan estas herramientas empresas para hacer desde procesos triviales hasta gestionar billones de eventos al d鱈a con latencias de milisegundos. Por supuesto, haremos una demo :)]]>
Trabajar en tiempo real con datos que se mueven muy r叩pido no es trivial, sobre todo con vol炭menes de datos elevados. Apache Flink y Apache BEAM est叩n espec鱈ficamente dise単adas para ese caso de uso. En esta charla te contar辿 los retos de la anal鱈tica en tiempo real, cu叩l es la arquitectura de Apache Flink, qu辿 es Apace BEAM, y c坦mo usan estas herramientas empresas para hacer desde procesos triviales hasta gestionar billones de eventos al d鱈a con latencias de milisegundos. Por supuesto, haremos una demo :)]]> Tue, 03 Nov 2020 16:30:35 GMT /slideshow/analitica-de-datos-en-tiempo-real-con-apache-flink-y-apache-beam/239070440 supercoco9@slideshare.net(supercoco9) Analitica de datos en tiempo real con Apache Flink y Apache BEAM supercoco9 Trabajar en tiempo real con datos que se mueven muy r叩pido no es trivial, sobre todo con vol炭menes de datos elevados. Apache Flink y Apache BEAM est叩n espec鱈ficamente dise単adas para ese caso de uso. En esta charla te contar辿 los retos de la anal鱈tica en tiempo real, cu叩l es la arquitectura de Apache Flink, qu辿 es Apace BEAM, y c坦mo usan estas herramientas empresas para hacer desde procesos triviales hasta gestionar billones de eventos al d鱈a con latencias de milisegundos. Por supuesto, haremos una demo :) <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/analiticadedatosentiemporealapacheflinkapachebeam-201103163035-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Trabajar en tiempo real con datos que se mueven muy r叩pido no es trivial, sobre todo con vol炭menes de datos elevados. Apache Flink y Apache BEAM est叩n espec鱈ficamente dise単adas para ese caso de uso. En esta charla te contar辿 los retos de la anal鱈tica en tiempo real, cu叩l es la arquitectura de Apache Flink, qu辿 es Apace BEAM, y c坦mo usan estas herramientas empresas para hacer desde procesos triviales hasta gestionar billones de eventos al d鱈a con latencias de milisegundos. Por supuesto, haremos una demo :)
Analitica de datos en tiempo real con Apache Flink y Apache BEAM from javier ramirez
]]> 217 0 https://cdn.slidesharecdn.com/ss_thumbnails/analiticadedatosentiemporealapacheflinkapachebeam-201103163035-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Getting started with streaming analytics /slideshow/getting-started-with-streaming-analytics/238188436 gettingstartedwithstreaminganalytics-part-1compressed-200824094537
In this webinar we explain which are some of the problems of streaming analytics, and why they are different to batch/big data analytics. Then we go into introducing some basic streaming concepts, like event queues, event processors, event vs processing time, and delivery guarantees. We end this first part of the series presenting a few of the most common open source components for streaming (Kafka, Spark, Flink, Cassandra, or ElasticSearch) and we mention the different options you have to run them on AWS. ]]>
In this webinar we explain which are some of the problems of streaming analytics, and why they are different to batch/big data analytics. Then we go into introducing some basic streaming concepts, like event queues, event processors, event vs processing time, and delivery guarantees. We end this first part of the series presenting a few of the most common open source components for streaming (Kafka, Spark, Flink, Cassandra, or ElasticSearch) and we mention the different options you have to run them on AWS. ]]> Mon, 24 Aug 2020 09:45:37 GMT /slideshow/getting-started-with-streaming-analytics/238188436 supercoco9@slideshare.net(supercoco9) Getting started with streaming analytics supercoco9 In this webinar we explain which are some of the problems of streaming analytics, and why they are different to batch/big data analytics. Then we go into introducing some basic streaming concepts, like event queues, event processors, event vs processing time, and delivery guarantees. We end this first part of the series presenting a few of the most common open source components for streaming (Kafka, Spark, Flink, Cassandra, or ElasticSearch) and we mention the different options you have to run them on AWS. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/gettingstartedwithstreaminganalytics-part-1compressed-200824094537-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> In this webinar we explain which are some of the problems of streaming analytics, and why they are different to batch/big data analytics. Then we go into introducing some basic streaming concepts, like event queues, event processors, event vs processing time, and delivery guarantees. We end this first part of the series presenting a few of the most common open source components for streaming (Kafka, Spark, Flink, Cassandra, or ElasticSearch) and we mention the different options you have to run them on AWS.
Getting started with streaming analytics from javier ramirez
]]> 225 0 https://cdn.slidesharecdn.com/ss_thumbnails/gettingstartedwithstreaminganalytics-part-1compressed-200824094537-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 https://cdn.slidesharecdn.com/profile-photo-supercoco9-48x48.jpg?cb=1743611689 As a Developer Advocate at QuestDB, I help developers make the most of their (fast) data, I make sure the core team behind QuestDB listens to every piece of feedback I get, and I facilitate collaboration in our open source repository. I love data storage, big and small. I have extensive experience with different SQL, NoSQL, graph, in-memory, and Big Data solutions. I like distributed, scalable, always-on systems. Before working at AWS I spent 20 years developing software professionally and sharing what I learnt with the community. I've spoken at events in more than 15 countries, mentored dozens of start-ups, taught for 6 years at universities, and trained hundreds of professional https//twitter.com/supercoco9 https://cdn.slidesharecdn.com/ss_thumbnails/futureoffastdbsbigdatalondon-241004154758-f94fea4d-thumbnail.jpg?width=320&height=320&fit=bounds slideshow/the-future-of-fast-databases-lessons-from-a-decade-of-questdb/272192275 The Future of Fast Dat... https://cdn.slidesharecdn.com/ss_thumbnails/questexactlyoncecommitconf-240704152526-ea886719-thumbnail.jpg?width=320&height=320&fit=bounds slideshow/como-hemos-implementado-semantica-de-exactly-once-en-nuestra-base-de-datos-de-alto-rendimiento/270065157 C坦mo hemos implementad... https://cdn.slidesharecdn.com/ss_thumbnails/howweaddedreplicationtoquestdb-240704152142-3a888164-thumbnail.jpg?width=320&height=320&fit=bounds slideshow/how-we-added-replication-to-questdb-jonthebeach/270065108 How We Added Replicati...