ºÝºÝߣshows by User: mvneves / http://www.slideshare.net/images/logo.gif ºÝºÝߣshows by User: mvneves / Wed, 07 Oct 2015 19:40:53 GMT ºÝºÝߣShare feed for ºÝºÝߣshows by User: mvneves MRemu: An Emulation-based Framework for Datacenter Network Experimentation using Realistic MapReduce Traffic /slideshow/mremu-an-emulationbased-framework-for-datacenter-network-experimentation-using-realistic-mapreduce-traffic/53663741 mascots2015-apresentacao-151007194053-lva1-app6891
As data volumes and the need for timely analysis grow, Big Data analytics frameworks have to scale out to hundred or even thousands of commodity servers. While such a scale-out is crucial to sustain desired computational throughput/latency and storage capacity, it comes at the cost of increased network traffic volumes and multiplicity of traffic patterns. Despite the sheer reality of the dependency between datacenter network (DCN) and time-to-insight through big data analysis, our experience as active networking researchers conveys that a large fraction of DCN research experimentation is conducted on network traces and/or synthetic flow traces. And while the respective results are often valuable as standalone contributions, in practice it turns out extremely difficult to quantitatively assess how the reported network optimization results translate to performance or fault-tolerance improvement for actual analytics runtimes, e.g., due to the ability of these runtimes to overlap communication with computation. This paper presents MRemu, an emulation-based framework for conducting reproducible datacenter network research using accurate MapReduce workloads and at system scales that are relevant to the size of target deployments, albeit without requiring access to a hardware infrastructure of such scale. We choose the MapReduce (MR) framework as a design point, for it is a common representative of the most widely deployed frameworks for analysis of large volumes of - structured and unstructured - data and is reported to be highly sensitive to network performance. With MRemu, it is possible to quantify the impact of various network design parameters and software-defined control techniques to key performance indicators of a given MR application. We show through targeted experimental validation that MRemu exhibits high fidelity, when compared to the performance of MR applications on a real scale-out cluster of 16 high-end servers. Also, as a proof of impact of our experimental framework, we showcase how we used MRemu to quantify the impact of network capacity among MR nodes to a selection of network-bound MR applications.]]>
As data volumes and the need for timely analysis grow, Big Data analytics frameworks have to scale out to hundred or even thousands of commodity servers. While such a scale-out is crucial to sustain desired computational throughput/latency and storage capacity, it comes at the cost of increased network traffic volumes and multiplicity of traffic patterns. Despite the sheer reality of the dependency between datacenter network (DCN) and time-to-insight through big data analysis, our experience as active networking researchers conveys that a large fraction of DCN research experimentation is conducted on network traces and/or synthetic flow traces. And while the respective results are often valuable as standalone contributions, in practice it turns out extremely difficult to quantitatively assess how the reported network optimization results translate to performance or fault-tolerance improvement for actual analytics runtimes, e.g., due to the ability of these runtimes to overlap communication with computation. This paper presents MRemu, an emulation-based framework for conducting reproducible datacenter network research using accurate MapReduce workloads and at system scales that are relevant to the size of target deployments, albeit without requiring access to a hardware infrastructure of such scale. We choose the MapReduce (MR) framework as a design point, for it is a common representative of the most widely deployed frameworks for analysis of large volumes of - structured and unstructured - data and is reported to be highly sensitive to network performance. With MRemu, it is possible to quantify the impact of various network design parameters and software-defined control techniques to key performance indicators of a given MR application. We show through targeted experimental validation that MRemu exhibits high fidelity, when compared to the performance of MR applications on a real scale-out cluster of 16 high-end servers. Also, as a proof of impact of our experimental framework, we showcase how we used MRemu to quantify the impact of network capacity among MR nodes to a selection of network-bound MR applications.]]> Wed, 07 Oct 2015 19:40:53 GMT /slideshow/mremu-an-emulationbased-framework-for-datacenter-network-experimentation-using-realistic-mapreduce-traffic/53663741 mvneves@slideshare.net(mvneves) MRemu: An Emulation-based Framework for Datacenter Network Experimentation using Realistic MapReduce Traffic mvneves As data volumes and the need for timely analysis grow, Big Data analytics frameworks have to scale out to hundred or even thousands of commodity servers. While such a scale-out is crucial to sustain desired computational throughput/latency and storage capacity, it comes at the cost of increased network traffic volumes and multiplicity of traffic patterns. Despite the sheer reality of the dependency between datacenter network (DCN) and time-to-insight through big data analysis, our experience as active networking researchers conveys that a large fraction of DCN research experimentation is conducted on network traces and/or synthetic flow traces. And while the respective results are often valuable as standalone contributions, in practice it turns out extremely difficult to quantitatively assess how the reported network optimization results translate to performance or fault-tolerance improvement for actual analytics runtimes, e.g., due to the ability of these runtimes to overlap communication with computation. This paper presents MRemu, an emulation-based framework for conducting reproducible datacenter network research using accurate MapReduce workloads and at system scales that are relevant to the size of target deployments, albeit without requiring access to a hardware infrastructure of such scale. We choose the MapReduce (MR) framework as a design point, for it is a common representative of the most widely deployed frameworks for analysis of large volumes of - structured and unstructured - data and is reported to be highly sensitive to network performance. With MRemu, it is possible to quantify the impact of various network design parameters and software-defined control techniques to key performance indicators of a given MR application. We show through targeted experimental validation that MRemu exhibits high fidelity, when compared to the performance of MR applications on a real scale-out cluster of 16 high-end servers. Also, as a proof of impact of our experimental framework, we showcase how we used MRemu to quantify the impact of network capacity among MR nodes to a selection of network-bound MR applications. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/mascots2015-apresentacao-151007194053-lva1-app6891-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> As data volumes and the need for timely analysis grow, Big Data analytics frameworks have to scale out to hundred or even thousands of commodity servers. While such a scale-out is crucial to sustain desired computational throughput/latency and storage capacity, it comes at the cost of increased network traffic volumes and multiplicity of traffic patterns. Despite the sheer reality of the dependency between datacenter network (DCN) and time-to-insight through big data analysis, our experience as active networking researchers conveys that a large fraction of DCN research experimentation is conducted on network traces and/or synthetic flow traces. And while the respective results are often valuable as standalone contributions, in practice it turns out extremely difficult to quantitatively assess how the reported network optimization results translate to performance or fault-tolerance improvement for actual analytics runtimes, e.g., due to the ability of these runtimes to overlap communication with computation. This paper presents MRemu, an emulation-based framework for conducting reproducible datacenter network research using accurate MapReduce workloads and at system scales that are relevant to the size of target deployments, albeit without requiring access to a hardware infrastructure of such scale. We choose the MapReduce (MR) framework as a design point, for it is a common representative of the most widely deployed frameworks for analysis of large volumes of - structured and unstructured - data and is reported to be highly sensitive to network performance. With MRemu, it is possible to quantify the impact of various network design parameters and software-defined control techniques to key performance indicators of a given MR application. We show through targeted experimental validation that MRemu exhibits high fidelity, when compared to the performance of MR applications on a real scale-out cluster of 16 high-end servers. Also, as a proof of impact of our experimental framework, we showcase how we used MRemu to quantify the impact of network capacity among MR nodes to a selection of network-bound MR applications.
MRemu: An Emulation-based Framework for Datacenter Network Experimentation using Realistic MapReduce Traffic from Marcelo Veiga Neves
]]> 475 6 https://cdn.slidesharecdn.com/ss_thumbnails/mascots2015-apresentacao-151007194053-lva1-app6891-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 A Performance Comparison of Container-based Virtualization Systems for MapReduce Clusters /slideshow/a-performance-comparison-of-containerbased-virtualization-systems-for-mapreduce-clusters/32098860 pdp2014-containers-slides-140309144403-phpapp01
]]>
]]> Sun, 09 Mar 2014 14:44:03 GMT /slideshow/a-performance-comparison-of-containerbased-virtualization-systems-for-mapreduce-clusters/32098860 mvneves@slideshare.net(mvneves) A Performance Comparison of Container-based Virtualization Systems for MapReduce Clusters mvneves <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/pdp2014-containers-slides-140309144403-phpapp01-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br>
A Performance Comparison of Container-based Virtualization Systems for MapReduce Clusters from Marcelo Veiga Neves
]]> 1818 2 https://cdn.slidesharecdn.com/ss_thumbnails/pdp2014-containers-slides-140309144403-phpapp01-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Impacto da Migra??o de M¨¢quinas Virtuais de Xen na Execu??o de Programas MPI - WSCAD 2007 https://pt.slideshare.net/slideshow/impacto-da-migrao-de-mquinas-virtuais-de-xen-na-execuo-de-programas-mpi-wscad-2007/15068298 wscad2007-veiga-121107095459-phpapp02
]]>
]]> Wed, 07 Nov 2012 09:54:55 GMT https://pt.slideshare.net/slideshow/impacto-da-migrao-de-mquinas-virtuais-de-xen-na-execuo-de-programas-mpi-wscad-2007/15068298 mvneves@slideshare.net(mvneves) Impacto da Migra??o de M¨¢quinas Virtuais de Xen na Execu??o de Programas MPI - WSCAD 2007 mvneves <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/wscad2007-veiga-121107095459-phpapp02-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br>
from Marcelo Veiga Neves
]]> 455 2 https://cdn.slidesharecdn.com/ss_thumbnails/wscad2007-veiga-121107095459-phpapp02-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Scheduling MapReduce Jobs in HPC Clusters /slideshow/scheduling-mapreduce-jobs-in-hpc-clusters-14973268/14973268 europar2012-neves-slides-121031164814-phpapp01
]]>
]]> Wed, 31 Oct 2012 16:48:12 GMT /slideshow/scheduling-mapreduce-jobs-in-hpc-clusters-14973268/14973268 mvneves@slideshare.net(mvneves) Scheduling MapReduce Jobs in HPC Clusters mvneves <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/europar2012-neves-slides-121031164814-phpapp01-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br>
Scheduling MapReduce Jobs in HPC Clusters from Marcelo Veiga Neves
]]> 1085 5 https://cdn.slidesharecdn.com/ss_thumbnails/europar2012-neves-slides-121031164814-phpapp01-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Linux em Sistemas Embarcados - SACTA 2010 - UNIPAMPA https://pt.slideshare.net/slideshow/linux-em-sistemas-embarcados/4673410 veiga-elinux-unipampa-100703134254-phpapp01
Palestra apresentada durante a Semana Acad¨ºmica do Centro de Tecnologia de Alegrete - SACTA 2010 - da Universidade Federal do Pampa - UNIPAMPA, em Alegrete - RS. Resumo: Esta palestra tem como tema o desenvolvimento de Linux para sistemas embarcados. Primeiramente, ser¨¢ apresentada uma vis?o geral da ¨¢rea. Na sequ¨ºncia, ser?o abordados os problemas t¨ªpicos envolvendo o porte do Linux para novas plataformas, a prepara??o de toolchains para cross-compila??o, bootloaders, escolha de sistemas de arquivos, prepara??o do kernel para trabalhar com os barramentos e dispositivos t¨ªpicos em sistemas embarcados, depura??o do hardware embarcado, entre outros. A palestra ter¨¢ tamb¨¦m uma se??o de demonstra??o utilizando uma placa baseada em PowerPC.]]>
Palestra apresentada durante a Semana Acad¨ºmica do Centro de Tecnologia de Alegrete - SACTA 2010 - da Universidade Federal do Pampa - UNIPAMPA, em Alegrete - RS. Resumo: Esta palestra tem como tema o desenvolvimento de Linux para sistemas embarcados. Primeiramente, ser¨¢ apresentada uma vis?o geral da ¨¢rea. Na sequ¨ºncia, ser?o abordados os problemas t¨ªpicos envolvendo o porte do Linux para novas plataformas, a prepara??o de toolchains para cross-compila??o, bootloaders, escolha de sistemas de arquivos, prepara??o do kernel para trabalhar com os barramentos e dispositivos t¨ªpicos em sistemas embarcados, depura??o do hardware embarcado, entre outros. A palestra ter¨¢ tamb¨¦m uma se??o de demonstra??o utilizando uma placa baseada em PowerPC.]]> Sat, 03 Jul 2010 13:42:43 GMT https://pt.slideshare.net/slideshow/linux-em-sistemas-embarcados/4673410 mvneves@slideshare.net(mvneves) Linux em Sistemas Embarcados - SACTA 2010 - UNIPAMPA mvneves Palestra apresentada durante a Semana Acad¨ºmica do Centro de Tecnologia de Alegrete - SACTA 2010 - da Universidade Federal do Pampa - UNIPAMPA, em Alegrete - RS. Resumo: Esta palestra tem como tema o desenvolvimento de Linux para sistemas embarcados. Primeiramente, ser¨¢ apresentada uma vis?o geral da ¨¢rea. Na sequ¨ºncia, ser?o abordados os problemas t¨ªpicos envolvendo o porte do Linux para novas plataformas, a prepara??o de toolchains para cross-compila??o, bootloaders, escolha de sistemas de arquivos, prepara??o do kernel para trabalhar com os barramentos e dispositivos t¨ªpicos em sistemas embarcados, depura??o do hardware embarcado, entre outros. A palestra ter¨¢ tamb¨¦m uma se??o de demonstra??o utilizando uma placa baseada em PowerPC. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/veiga-elinux-unipampa-100703134254-phpapp01-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Palestra apresentada durante a Semana Acad¨ºmica do Centro de Tecnologia de Alegrete - SACTA 2010 - da Universidade Federal do Pampa - UNIPAMPA, em Alegrete - RS. Resumo: Esta palestra tem como tema o desenvolvimento de Linux para sistemas embarcados. Primeiramente, ser¨¢ apresentada uma vis?o geral da ¨¢rea. Na sequ¨ºncia, ser?o abordados os problemas t¨ªpicos envolvendo o porte do Linux para novas plataformas, a prepara??o de toolchains para cross-compila??o, bootloaders, escolha de sistemas de arquivos, prepara??o do kernel para trabalhar com os barramentos e dispositivos t¨ªpicos em sistemas embarcados, depura??o do hardware embarcado, entre outros. A palestra ter¨¢ tamb¨¦m uma se??o de demonstra??o utilizando uma placa baseada em PowerPC.
from Marcelo Veiga Neves
]]> 867 13 https://cdn.slidesharecdn.com/ss_thumbnails/veiga-elinux-unipampa-100703134254-phpapp01-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 Desenvolvendo Sistemas de Linux Embarcado - Tchelinux 2009 https://pt.slideshare.net/slideshow/desenvolvendo-sistemas-de-linux-embarcado-tchelinux-2009/4673396 veiga-elinux-tchelinux2009-100703133638-phpapp01
Palestra apresentada no evento Tchelinux 2009 em Porto Alegre. Resumo da Palestra: Esta palestra abordar¨¢ os itens necess¨¢rios para construir um sistema de Linux embarcado. Ser?o abordados os problemas t¨ªpicos envolvendo Linux em sistemas embarcados, a prepara??o de toolchains para cross-compila??o, bootloaders, escolha de sistemas de arquivos, prepara??o do kernel para trabalhar com os barramentos e dispositivos t¨ªpicos em sistemas embarcados, depura??o do hardware embarcado, entre outros. A palestra ter¨¢ tamb¨¦m uma se??o de demonstra??o utilizando uma placa de desenvolvimento baseada em PowerPC. ]]>
Palestra apresentada no evento Tchelinux 2009 em Porto Alegre. Resumo da Palestra: Esta palestra abordar¨¢ os itens necess¨¢rios para construir um sistema de Linux embarcado. Ser?o abordados os problemas t¨ªpicos envolvendo Linux em sistemas embarcados, a prepara??o de toolchains para cross-compila??o, bootloaders, escolha de sistemas de arquivos, prepara??o do kernel para trabalhar com os barramentos e dispositivos t¨ªpicos em sistemas embarcados, depura??o do hardware embarcado, entre outros. A palestra ter¨¢ tamb¨¦m uma se??o de demonstra??o utilizando uma placa de desenvolvimento baseada em PowerPC. ]]> Sat, 03 Jul 2010 13:36:30 GMT https://pt.slideshare.net/slideshow/desenvolvendo-sistemas-de-linux-embarcado-tchelinux-2009/4673396 mvneves@slideshare.net(mvneves) Desenvolvendo Sistemas de Linux Embarcado - Tchelinux 2009 mvneves Palestra apresentada no evento Tchelinux 2009 em Porto Alegre. Resumo da Palestra: Esta palestra abordar¨¢ os itens necess¨¢rios para construir um sistema de Linux embarcado. Ser?o abordados os problemas t¨ªpicos envolvendo Linux em sistemas embarcados, a prepara??o de toolchains para cross-compila??o, bootloaders, escolha de sistemas de arquivos, prepara??o do kernel para trabalhar com os barramentos e dispositivos t¨ªpicos em sistemas embarcados, depura??o do hardware embarcado, entre outros. A palestra ter¨¢ tamb¨¦m uma se??o de demonstra??o utilizando uma placa de desenvolvimento baseada em PowerPC. <img style="border:1px solid #C3E6D8;float:right;" alt="" src="https://cdn.slidesharecdn.com/ss_thumbnails/veiga-elinux-tchelinux2009-100703133638-phpapp01-thumbnail.jpg?width=120&amp;height=120&amp;fit=bounds" /><br> Palestra apresentada no evento Tchelinux 2009 em Porto Alegre. Resumo da Palestra: Esta palestra abordar¨¢ os itens necess¨¢rios para construir um sistema de Linux embarcado. Ser?o abordados os problemas t¨ªpicos envolvendo Linux em sistemas embarcados, a prepara??o de toolchains para cross-compila??o, bootloaders, escolha de sistemas de arquivos, prepara??o do kernel para trabalhar com os barramentos e dispositivos t¨ªpicos em sistemas embarcados, depura??o do hardware embarcado, entre outros. A palestra ter¨¢ tamb¨¦m uma se??o de demonstra??o utilizando uma placa de desenvolvimento baseada em PowerPC.
from Marcelo Veiga Neves
]]> 658 13 https://cdn.slidesharecdn.com/ss_thumbnails/veiga-elinux-tchelinux2009-100703133638-phpapp01-thumbnail.jpg?width=120&height=120&fit=bounds presentation Black http://activitystrea.ms/schema/1.0/post http://activitystrea.ms/schema/1.0/posted 0 https://cdn.slidesharecdn.com/profile-photo-mvneves-48x48.jpg?cb=1666166475 ... marceloneves.net https://cdn.slidesharecdn.com/ss_thumbnails/mascots2015-apresentacao-151007194053-lva1-app6891-thumbnail.jpg?width=320&height=320&fit=bounds slideshow/mremu-an-emulationbased-framework-for-datacenter-network-experimentation-using-realistic-mapreduce-traffic/53663741 MRemu: An Emulation-ba... https://cdn.slidesharecdn.com/ss_thumbnails/pdp2014-containers-slides-140309144403-phpapp01-thumbnail.jpg?width=320&height=320&fit=bounds slideshow/a-performance-comparison-of-containerbased-virtualization-systems-for-mapreduce-clusters/32098860 A Performance Comparis... https://cdn.slidesharecdn.com/ss_thumbnails/wscad2007-veiga-121107095459-phpapp02-thumbnail.jpg?width=320&height=320&fit=bounds slideshow/impacto-da-migrao-de-mquinas-virtuais-de-xen-na-execuo-de-programas-mpi-wscad-2007/15068298 Impacto da Migra??o de...