Kognitio overview jan 2013
?
0 likes?130 views
Kognitio provides an in-memory analytical platform that loads large datasets entirely into RAM, allowing clients to perform complex analytics on datasets as large as tens of billions of records in just tens of seconds. The platform utilizes massively parallel processing across commodity servers to scale linearly with no single points of failure. Kognitio works with clients across industries to power solutions for media analytics, customer loyalty programs, online travel, mobile advertising and more.
![Not Only SQL: any language in-line
Kognitio External Scripts
C Run third party binaries or scripts embedded within SQL
? Perl, Python, Java, R, SAS, etc.
? One-to-many rows in, zero-to-many rows out, one to one
create interpreter perlinterp
command '/usr/bin/perl' sends 'csv' receives 'csv' ;
select top 1000 words, count(*) This reads long comments
from (external script using environment perlinterp text from customer enquiry
receives (txt varchar(32000))
sends (words varchar(100)) table, in line perl converts
script S'endofperl( long text into output
while(<>)
{ stream of words (one word
chomp(); per row), query selects top
s/[,.!_]//g;
foreach $c (split(/ /)) 1000 words by frequency
{ if($c =~ /^[a-zA-Z]+$/) { print "$cn￣} } using standard SQL
}
)endofperl' aggregation
from (select comments from customer_enquiry))dt
group by 1
order by 2 desc;](https://image.slidesharecdn.com/kognitiooverviewjan2013-130125090422-phpapp01/85/Kognitio-overview-jan-2013-11-320.jpg)
Kognitio overview jan 2013
- 1. Kognitio Analytical Platform Company Update December 2012
- 2. Kognitio Kognitio is focused on providing the premier high- performance analytical platform to power business insight around the world. ?Privately?held ?Dev Labs?in?the?UK? ?Leadership??in?US ?~100?employees Core?product: ?MPP?in\memory? analytical?platform ?Built?from?the? ground\up?to?satisfy? large?and?complex? analytics?on?big?data? sets
- 3. WW Client Growth *Some clients NDA
- 4. Brand Streamlining C New Kognitio Product Hierarchy Console Kognitio v8 Software MDX Connector Admin Tools Analytical Processing Cube Designer Accelerator for Hadoop Excel Add-in Accelerator for ´ Analytical Appliance
- 5. Flexibility Engrained to our Business Model Public Cloud (SaaS) Private Cloud Software Appliance Partnerships Low costs, no CapEx Pre-built and configured Industry-standard x86 Commodity hardware ISVs: requirement, immediate Highest security for Linux servers per client preference specialized/industry provisioning sensititive data sets Rapid deployment and Bespoke formula of solutions Incur charged per hour Hosted in Tier-3 data implementation RAM data memory, Services: on demand (CPU core/ centers via specialized Typical license- server cores and disk implementation and hour) hosting providers maintenance contract specifications delivery Provided by Kognitio, ~ 48 hour provisioning for customers Profit = software Distributors: expanded hosted by Amazon Web Massively scalable license + hardware market coverage Services (AWS) margins - Flexible delivery model to meet client requirements - Partnership channel builds ecosystem and expands reach - Revenue model: - One-time charge: volume-based software licenses - Recurring revenue: maintenance and support
- 6. Analytical Platform Reference Architecture
- 7. In-memory Analytical Platform ＾pull very large amounts of data from existing data storage (persistence) systems into high speed computer memory￣ C can be existing traditional disk based data warehouse products, operational systems, Kognitio¨s own disk subsystem or increasingly distributed parallel file systems such as Hadoop or cloud storage ? Scale the power as required ? Adaptable capacity C Scale up / down as when needed within server farm ? Utilize local disk for near-line store of regularly used reference data or result sets
- 8. What is an ＾In-memory￣ Analytical Platform? ? A database where all of the data of interest or specific portions of the data have been permanently pre-loaded into a computers random access memory (RAM). ? Not a large cache C Data is held in structures that take advantage of the properties of RAM C NOT copies of frequently used disk blocks C The databases query optimiser knows at all times exactly which data is in memory and which is not
- 9. Speed & Scale from ＾True MPP￣ ? Memory & CPU on an individual server = NOWHERE near enough for big data C Moore¨s Law C The power of a processor doubles every two years C Data volumes C Double every year!! ? The only way to keep up is to parallelise or scale-out ? Combine the RAM of many individual servers Many ? ? many CPU cores spread across many CPUs, housed in ? many individual computers (1 to 1000+) C Data is split across all the CPU cores C All database operations are parallelised with no points of serialisation C This is true MPP ? Every CPU core in Every ? Every server needs to efficiently involved in ? Every query
- 10. V8 Enables the Analytical Platform Reference Architecture External Functions Not Only SQL External Tables Kognitio Storage as an External table Hadoop Connector Other Connectors
- 11. Not Only SQL: any language in-line Kognitio External Scripts C Run third party binaries or scripts embedded within SQL ? Perl, Python, Java, R, SAS, etc. ? One-to-many rows in, zero-to-many rows out, one to one create interpreter perlinterp command '/usr/bin/perl' sends 'csv' receives 'csv' ; select top 1000 words, count(*) This reads long comments from (external script using environment perlinterp text from customer enquiry receives (txt varchar(32000)) sends (words varchar(100)) table, in line perl converts script S'endofperl( long text into output while(<>) { stream of words (one word chomp(); per row), query selects top s/[,.!_]//g; foreach $c (split(/ /)) 1000 words by frequency { if($c =~ /^[a-zA-Z]+$/) { print "$cn￣} } using standard SQL } )endofperl' aggregation from (select comments from customer_enquiry))dt group by 1 order by 2 desc;
- 12. Kognitio Hadoop Connectors HDFS Connector ? Connector defines access to hdfs file system ? External table accesses row-based data in hdfs ? Dynamic access or ＾pin￣ data into memory ? Complete hdfs file is loaded into memory Filter Agent Connector ? Connector uploads agent to Hadoop nodes ? Query passes selections and relevant predicates to agent ? Data filtering and projection takes place locally on each Hadoop node ? Only data of interest in loaded into memory via parallel load streams
- 13. Innovative client solutions TiVo?Research?&?Analytics?40?TBs?of?RAM?that?perform?complex?media?analytics,? cross\correlating?data?from?over?22?sources?with?set\top?box?data?to?allow? Software advertisers,?networks?and?agencies??to?analyze?the?ROI?of?creative?campaigns? while?they?are?still?in?flight,?enabling?self\service?reporting?for?business?users The?VivaKi Nerve?Center?provides?social?media?and?other?analytics?for??campaign? Public? monitoring?and?near?real\time?advertising?effectiveness.??This?enables?agencies?in?the? Cloud Publicis Global?Network?to?provide?deep\dive?analytics?into?TBs?of?data?in?seconds AIMIA?provides?self\service?customer?loyalty?analysis?on?over?24?billion?transactions? that?are?live?in\memory?full?volumes?of?POS?data.??Retailers,?Customer?Packaged?Goods? Appliance companies?and?other?service?providers,?provide?merchandise?managers?with??＾train\of\ thought￣?analysis?to?better?target?customers. Orbitz?leverages?Kognitio?Cloud?to?take?large?volumes?of?complex?data,?ingested?in?real? Private? time?from?web?channels,?demographic?and?psychographic?data,?customer?segmentation? Cloud and?modeling?scores?and?turn?it?into?actionable?intelligence,?allowing?them?to?think?of?new? ways?of?offering?the?right?products?and?services?to?its?current?and?prospective?client?base. PlaceIQ provdes actionable?hyper\local?Mobile?BI?location?intelligence.??They? leverage?Kognitio?to?extracts?intelligence?from?large?amounts?of?place,?social?and? Public? mobile?location\based?data?to?create?hyper\local,?targetable?audience?profiles,? Cloud giving?advertisers?the?power?to?connect?with?consumers?at?the?right?place,?at?the? right?time,?with?the?right?message.?
- 14. Analytics on tens of billions of events in tens of seconds with NO DBA Context?for?media?analytics:? ? In\memory?analytical?database?for?Big?Data ? Correlate?everything?to?everything ? MPP?+?Linear?Scalability ? Predictable?and?ultra\fast?performance Challenges ? >?22?data?sources C Expanding?volumes?of?data ? Commodity?servers/equipment C Few?opportunities?for? summarization?(demographics,? ? Market\available?IT?skills purchaser?targets,?etc.) ? No?solution?re\engineering C Data?too?large/complex?for? traditional?database?systems C Need?for?simple?administration Solution?Benefits Mars,?Inc.:? C Reports?allow?advertisers,?networks?and?agencies??to?analyze?the? ＾By?using?TRA?to?improve?media?plans,?creative?and? relative?strengths?and?weaknesses?of?different?creative? flighting,?Mars?has?achieved?a?portfolio?increase?in?ROI? executions,?and?how?such?variables?as?program?environment,? versus?a?year?ago?of?25% in?one?category?and?35% in?a? time?slots,?and?pod?position?impact?their?ROI second?category.￣ C Enables?self\service?reporting?for?business?users
- 15. Thank You! connect contact Michael Hiskey kognitio.com Vice President, Marketing & Business Development kognitio.tel michael.hiskey@kognitio.com +1.917.375.8196 kognitio.com/blog Paul Groom VP, Business Intelligence Paul.groom@kognitio.com twitter.com/kognitio John Coppins linkedin.com/companies/kognitio SVP, Kognitio Cloud john/.coppins@kognitio.com tinyurl.com/kognitio Steve Friedberg MMI Communications youtube.com/kognitio steve@mmicommunications.com