際際滷

際際滷Share a Scribd company logo
Plone on RelStorage
Calvin Hendryx-Parker
PLONE CONFERENCE 2010
Saturday, October 30, 2010
PLONE CONFERENCE 2010Overview
 How does Zope store data
 Why RelStorage?
 How to implement RelStorage
 Some reasons not to use RelStorage
Saturday, October 30, 2010
PLONE CONFERENCE 2010Zope Storage
Methods
Saturday, October 30, 2010
PLONE CONFERENCE 2010What is the ZODB?
 Object oriented database
 Used by Zope to hold everything from content
to scripts, to service con鍖gurations
Saturday, October 30, 2010
PLONE CONFERENCE 2010What is Filestorage?
 The default backend for Zope/ZEO
 var/鍖lestorage/Data.fs
 Does an in-memory index at startup
 Data appends to the end
 Back-up utility is Repozo
Saturday, October 30, 2010
PLONE CONFERENCE 2010What is RelStorage?
 Drop in replacement
 Replaces ZEO
 Eliminates Data.fs
 Stores pickles
 Is still ZODB
 Uses relational
databases
Saturday, October 30, 2010
PLONE CONFERENCE 2010Why Change?
Saturday, October 30, 2010
PLONE CONFERENCE 2010Advantages of
RelStorage
 Existing infrastructure
 Existing expertise
 Wide range of management tools
 High availability, high performance clusters
 No in-memory index at startup
 Scales well as your database grows
 High Concurrency
Saturday, October 30, 2010
PLONE CONFERENCE 2010Real World
 Major Health System Site
 Java portion of site locked up
 Restarted Serivce with Little Impact
Saturday, October 30, 2010
PLONE CONFERENCE 2010Databases Supported
 MySQL 5.0.32 or better
 MySQLdb >= 1.2.2
 PostgreSQL 8.1 or
better
 psycopg2
 Oracle 10g
 cx_Oracle
Saturday, October 30, 2010
PLONE CONFERENCE 2010How Do I Install It?
 For MySQL add the
following to buildout.cfg:
eggs =
RelStorage
MySQL-Python
fake-eggs =
ZODB3 = 3.7.1
[instance]
type mysql
db plone
user plone
host localhost
password terriblepassword
Saturday, October 30, 2010
PLONE CONFERENCE 2010How Do I Install It?
 For PostgreSQL add the
following to buildout.cfg:
eggs =
RelStorage
fake-eggs =
ZODB3 = 3.7.1
psycopg2
[instance]
type postgresql
dbname plone
user plone
host localhost
password terriblepassword
Saturday, October 30, 2010
PLONE CONFERENCE 2010Options
 blob-dir
 shared-blob-dir
 keep-history
 replica-conf
 pack-gc
 pack-dry-run
 cache-servers
Saturday, October 30, 2010
PLONE CONFERENCE 2010Available Tools
 zodbconvert
 Converts from 鍖lestorage to {mysql/pgsql/oracle}
 Convert from {mysql/pgsql/oracle} to 鍖lestorage
 Convert from one Database to another
(ie: mysql to oracle)
 zodbpack
 Allows concurrent connections
 Use from cron
Saturday, October 30, 2010
PLONE CONFERENCE 2010zodbconvert.conf
<filestorage source>
path /opt/default/zinstance/var/
filestorage/Data.fs
</filestorage>
<relstorage destination>
<mysql>
db plone
user plone
passwd terriblepassword
</mysql>
</relstorage>
Converting from 鍖lestorage to MySQL
Saturday, October 30, 2010
PLONE CONFERENCE 2010Running zodbconvert
$ sudo /opt/mysql/zinstance/bin/zodbconvert --clear
zodbconvert.conf
Storages opened successfully.
All transactions copied successfully.
Saturday, October 30, 2010
PLONE CONFERENCE 2010Whats the Catch?
Saturday, October 30, 2010
PLONE CONFERENCE 2010There Are Pitfalls
 Added complexity
 Additional server load
 Additional memory utilization
 The database is full of pickles, not plain text
 Your database still needs to be packed
Saturday, October 30, 2010
PLONE CONFERENCE 2010What We Talked
About
 Zope storage engines: ZEO and RelStorage
 Why you might choose RelStorage
 How to install RelStorage
 Tools for RelStorage
 Why you might not use RelStorage
Saturday, October 30, 2010
PLONE CONFERENCE 2010Links
 http://pypi.python.org/pypi/RelStorage
 http://shane.willowrise.com/archives/relstorage-1-4-0b1-and-zodbshootout
Saturday, October 30, 2010
PLONE CONFERENCE 2010Credits:
 http://www.鍖ickr.com/photos/kaplanbr/
2465380377/
 http://www.鍖ickr.com/photos/
shuttercat7/416008944/
 http://www.鍖ickr.com/photos/juniorvelo/
3267647833/
 http://www.鍖ickr.com/photos/
arthurjohnpicton/4383221264/
 http://www.鍖ickr.com/photos/generated/
3249283352/
Saturday, October 30, 2010
Check out
sixfeetup.com/demos
Saturday, October 30, 2010
Ad

Recommended

Enterprise search in plone using solr
Enterprise search in plone using solr
Calvin Hendryx-Parker
Enterprise search in Plone using Solr
Enterprise search in Plone using Solr
Calvin Hendryx-Parker
QueryPath: It's like PHP jQuery in Drupal!
QueryPath: It's like PHP jQuery in Drupal!
Matt Butcher
QueryPath, Mash-ups, and Web Services
QueryPath, Mash-ups, and Web Services
Matt Butcher
仂仍仂亟亳仄亳 仂亳从. Getting maximum of python, django with postgres 9.4. PyCon B...
仂仍仂亟亳仄亳 仂亳从. Getting maximum of python, django with postgres 9.4. PyCon B...
Alina Dolgikh
Teets, "NISO Next Generation Discovery"
Teets, "NISO Next Generation Discovery"
National Information Standards Organization (NISO)
Sphinx - High performance full-text search for MySQL
Sphinx - High performance full-text search for MySQL
Nguyen Van Vuong
Gizzard, DAL and more
Gizzard, DAL and more
fulin tang
Pse2010 rel storage
Pse2010 rel storage
Lars Noldan
RelStorage Plone Zope RDB Storage Backend
RelStorage Plone Zope RDB Storage Backend
Jens Klein
ZODB, the Zope Object Database (May 2003)
ZODB, the Zope Object Database (May 2003)
Kiran Jonnalagadda
Intro to Big Data - Orlando Code Camp 2014
Intro to Big Data - Orlando Code Camp 2014
John Ternent
Ungooglable
Ungooglable
Elizabeth Leddy
ZODB Tips and Tricks
ZODB Tips and Tricks
Carlos de la Guardia
ZODB Tips and Tricks
ZODB Tips and Tricks
Jazkarta, Inc.
Storage Methods for Nonstandard Data Patterns
Storage Methods for Nonstandard Data Patterns
Bob Burgess
Jared Whitlock Open Source In The Enterprise Plone @ Novell
Jared Whitlock Open Source In The Enterprise Plone @ Novell
Vincenzo Barone
Why Plone Will Die
Why Plone Will Die
Andreas Jung
Some dope on Zope (Jan 2002, Bangalore LUG)
Some dope on Zope (Jan 2002, Bangalore LUG)
Kiran Jonnalagadda
How to be a Postgres DBA in a Pinch
How to be a Postgres DBA in a Pinch
ElizabethGarrettChri
20080528dublinpt1
20080528dublinpt1
Jeff Hammerbacher
Build Python CMS The Plone Way
Build Python CMS The Plone Way
TsungWei Hu
Content Mirror
Content Mirror
fravy
Big Data (NJ SQL Server User Group)
Big Data (NJ SQL Server User Group)
Don Demcsak
PostgreSQL + ZFS best practices
PostgreSQL + ZFS best practices
Sean Chittenden
PostgreSQL on ZFS Lightning Talk
PostgreSQL on ZFS Lightning Talk
Sean Chittenden
NoSQL in the context of Social Web
NoSQL in the context of Social Web
Bogdan Gaza
Big Data @ Bodensee Barcamp 2010
Big Data @ Bodensee Barcamp 2010
c1sc0
Blending AI in Enterprise Architecture.pdf
Blending AI in Enterprise Architecture.pdf
Calvin Hendryx-Parker
Plone and Drupal -- CMS Coexistance in Higher Education
Plone and Drupal -- CMS Coexistance in Higher Education
Calvin Hendryx-Parker

More Related Content

Similar to Plone on RelStorage (20)

Pse2010 rel storage
Pse2010 rel storage
Lars Noldan
RelStorage Plone Zope RDB Storage Backend
RelStorage Plone Zope RDB Storage Backend
Jens Klein
ZODB, the Zope Object Database (May 2003)
ZODB, the Zope Object Database (May 2003)
Kiran Jonnalagadda
Intro to Big Data - Orlando Code Camp 2014
Intro to Big Data - Orlando Code Camp 2014
John Ternent
Ungooglable
Ungooglable
Elizabeth Leddy
ZODB Tips and Tricks
ZODB Tips and Tricks
Carlos de la Guardia
ZODB Tips and Tricks
ZODB Tips and Tricks
Jazkarta, Inc.
Storage Methods for Nonstandard Data Patterns
Storage Methods for Nonstandard Data Patterns
Bob Burgess
Jared Whitlock Open Source In The Enterprise Plone @ Novell
Jared Whitlock Open Source In The Enterprise Plone @ Novell
Vincenzo Barone
Why Plone Will Die
Why Plone Will Die
Andreas Jung
Some dope on Zope (Jan 2002, Bangalore LUG)
Some dope on Zope (Jan 2002, Bangalore LUG)
Kiran Jonnalagadda
How to be a Postgres DBA in a Pinch
How to be a Postgres DBA in a Pinch
ElizabethGarrettChri
20080528dublinpt1
20080528dublinpt1
Jeff Hammerbacher
Build Python CMS The Plone Way
Build Python CMS The Plone Way
TsungWei Hu
Content Mirror
Content Mirror
fravy
Big Data (NJ SQL Server User Group)
Big Data (NJ SQL Server User Group)
Don Demcsak
PostgreSQL + ZFS best practices
PostgreSQL + ZFS best practices
Sean Chittenden
PostgreSQL on ZFS Lightning Talk
PostgreSQL on ZFS Lightning Talk
Sean Chittenden
NoSQL in the context of Social Web
NoSQL in the context of Social Web
Bogdan Gaza
Big Data @ Bodensee Barcamp 2010
Big Data @ Bodensee Barcamp 2010
c1sc0
Pse2010 rel storage
Pse2010 rel storage
Lars Noldan
RelStorage Plone Zope RDB Storage Backend
RelStorage Plone Zope RDB Storage Backend
Jens Klein
ZODB, the Zope Object Database (May 2003)
ZODB, the Zope Object Database (May 2003)
Kiran Jonnalagadda
Intro to Big Data - Orlando Code Camp 2014
Intro to Big Data - Orlando Code Camp 2014
John Ternent
ZODB Tips and Tricks
ZODB Tips and Tricks
Jazkarta, Inc.
Storage Methods for Nonstandard Data Patterns
Storage Methods for Nonstandard Data Patterns
Bob Burgess
Jared Whitlock Open Source In The Enterprise Plone @ Novell
Jared Whitlock Open Source In The Enterprise Plone @ Novell
Vincenzo Barone
Why Plone Will Die
Why Plone Will Die
Andreas Jung
Some dope on Zope (Jan 2002, Bangalore LUG)
Some dope on Zope (Jan 2002, Bangalore LUG)
Kiran Jonnalagadda
How to be a Postgres DBA in a Pinch
How to be a Postgres DBA in a Pinch
ElizabethGarrettChri
Build Python CMS The Plone Way
Build Python CMS The Plone Way
TsungWei Hu
Content Mirror
Content Mirror
fravy
Big Data (NJ SQL Server User Group)
Big Data (NJ SQL Server User Group)
Don Demcsak
PostgreSQL + ZFS best practices
PostgreSQL + ZFS best practices
Sean Chittenden
PostgreSQL on ZFS Lightning Talk
PostgreSQL on ZFS Lightning Talk
Sean Chittenden
NoSQL in the context of Social Web
NoSQL in the context of Social Web
Bogdan Gaza
Big Data @ Bodensee Barcamp 2010
Big Data @ Bodensee Barcamp 2010
c1sc0

More from Calvin Hendryx-Parker (9)

Blending AI in Enterprise Architecture.pdf
Blending AI in Enterprise Architecture.pdf
Calvin Hendryx-Parker
Plone and Drupal -- CMS Coexistance in Higher Education
Plone and Drupal -- CMS Coexistance in Higher Education
Calvin Hendryx-Parker
Plone roadmap
Plone roadmap
Calvin Hendryx-Parker
How to seal the deal
How to seal the deal
Calvin Hendryx-Parker
2010 py ohio supervisor talk
2010 py ohio supervisor talk
Calvin Hendryx-Parker
Social Networking Tools Session Three
Social Networking Tools Session Three
Calvin Hendryx-Parker
Social Networking Tools Session One
Social Networking Tools Session One
Calvin Hendryx-Parker
Social Networking Tools Session Two
Social Networking Tools Session Two
Calvin Hendryx-Parker
Plone's Anatomy
Plone's Anatomy
Calvin Hendryx-Parker
Ad

Recently uploaded (20)

AI vs Human Writing: Can You Tell the Difference?
AI vs Human Writing: Can You Tell the Difference?
Shashi Sathyanarayana, Ph.D
Down the Rabbit Hole Solving 5 Training Roadblocks
Down the Rabbit Hole Solving 5 Training Roadblocks
Rustici Software
AudGram Review: Build Visually Appealing, AI-Enhanced Audiograms to Engage Yo...
AudGram Review: Build Visually Appealing, AI-Enhanced Audiograms to Engage Yo...
SOFTTECHHUB
Bridging the divide: A conversation on tariffs today in the book industry - T...
Bridging the divide: A conversation on tariffs today in the book industry - T...
BookNet Canada
FIDO Seminar: Targeting Trust: The Future of Identity in the Workforce.pptx
FIDO Seminar: Targeting Trust: The Future of Identity in the Workforce.pptx
FIDO Alliance
Artificial Intelligence in the Nonprofit Boardroom.pdf
Artificial Intelligence in the Nonprofit Boardroom.pdf
OnBoard
Key Requirements to Successfully Implement Generative AI in Edge DevicesOpt...
Key Requirements to Successfully Implement Generative AI in Edge DevicesOpt...
Edge AI and Vision Alliance
FIDO Alliance Seminar State of Passkeys.pptx
FIDO Alliance Seminar State of Passkeys.pptx
FIDO Alliance
Edge-banding-machines-edgeteq-s-200-en-.pdf
Edge-banding-machines-edgeteq-s-200-en-.pdf
AmirStern2
Creating Inclusive Digital Learning with AI: A Smarter, Fairer Future
Creating Inclusive Digital Learning with AI: A Smarter, Fairer Future
Impelsys Inc.
TrustArc Webinar - 2025 Global Privacy Survey
TrustArc Webinar - 2025 Global Privacy Survey
TrustArc
Reducing Conflicts and Increasing Safety Along the Cycling Networks of East-F...
Reducing Conflicts and Increasing Safety Along the Cycling Networks of East-F...
Safe Software
AI VIDEO MAGAZINE - June 2025 - r/aivideo
AI VIDEO MAGAZINE - June 2025 - r/aivideo
1pcity Studios, Inc
From Enterprise to Makers: Driving Vision AI Innovation at the Extreme Edge,...
From Enterprise to Makers: Driving Vision AI Innovation at the Extreme Edge,...
Edge AI and Vision Alliance
Enabling BIM / GIS integrations with Other Systems with FME
Enabling BIM / GIS integrations with Other Systems with FME
Safe Software
Kubernetes Security Act Now Before Its Too Late
Kubernetes Security Act Now Before Its Too Late
Michael Furman
Data Validation and System Interoperability
Data Validation and System Interoperability
Safe Software
June Patch Tuesday
June Patch Tuesday
Ivanti
National Fuels Treatments Initiative: Building a Seamless Map of Hazardous Fu...
National Fuels Treatments Initiative: Building a Seamless Map of Hazardous Fu...
Safe Software
War_And_Cyber_3_Years_Of_Struggle_And_Lessons_For_Global_Security.pdf
War_And_Cyber_3_Years_Of_Struggle_And_Lessons_For_Global_Security.pdf
biswajitbanerjee38
AI vs Human Writing: Can You Tell the Difference?
AI vs Human Writing: Can You Tell the Difference?
Shashi Sathyanarayana, Ph.D
Down the Rabbit Hole Solving 5 Training Roadblocks
Down the Rabbit Hole Solving 5 Training Roadblocks
Rustici Software
AudGram Review: Build Visually Appealing, AI-Enhanced Audiograms to Engage Yo...
AudGram Review: Build Visually Appealing, AI-Enhanced Audiograms to Engage Yo...
SOFTTECHHUB
Bridging the divide: A conversation on tariffs today in the book industry - T...
Bridging the divide: A conversation on tariffs today in the book industry - T...
BookNet Canada
FIDO Seminar: Targeting Trust: The Future of Identity in the Workforce.pptx
FIDO Seminar: Targeting Trust: The Future of Identity in the Workforce.pptx
FIDO Alliance
Artificial Intelligence in the Nonprofit Boardroom.pdf
Artificial Intelligence in the Nonprofit Boardroom.pdf
OnBoard
Key Requirements to Successfully Implement Generative AI in Edge DevicesOpt...
Key Requirements to Successfully Implement Generative AI in Edge DevicesOpt...
Edge AI and Vision Alliance
FIDO Alliance Seminar State of Passkeys.pptx
FIDO Alliance Seminar State of Passkeys.pptx
FIDO Alliance
Edge-banding-machines-edgeteq-s-200-en-.pdf
Edge-banding-machines-edgeteq-s-200-en-.pdf
AmirStern2
Creating Inclusive Digital Learning with AI: A Smarter, Fairer Future
Creating Inclusive Digital Learning with AI: A Smarter, Fairer Future
Impelsys Inc.
TrustArc Webinar - 2025 Global Privacy Survey
TrustArc Webinar - 2025 Global Privacy Survey
TrustArc
Reducing Conflicts and Increasing Safety Along the Cycling Networks of East-F...
Reducing Conflicts and Increasing Safety Along the Cycling Networks of East-F...
Safe Software
AI VIDEO MAGAZINE - June 2025 - r/aivideo
AI VIDEO MAGAZINE - June 2025 - r/aivideo
1pcity Studios, Inc
From Enterprise to Makers: Driving Vision AI Innovation at the Extreme Edge,...
From Enterprise to Makers: Driving Vision AI Innovation at the Extreme Edge,...
Edge AI and Vision Alliance
Enabling BIM / GIS integrations with Other Systems with FME
Enabling BIM / GIS integrations with Other Systems with FME
Safe Software
Kubernetes Security Act Now Before Its Too Late
Kubernetes Security Act Now Before Its Too Late
Michael Furman
Data Validation and System Interoperability
Data Validation and System Interoperability
Safe Software
June Patch Tuesday
June Patch Tuesday
Ivanti
National Fuels Treatments Initiative: Building a Seamless Map of Hazardous Fu...
National Fuels Treatments Initiative: Building a Seamless Map of Hazardous Fu...
Safe Software
War_And_Cyber_3_Years_Of_Struggle_And_Lessons_For_Global_Security.pdf
War_And_Cyber_3_Years_Of_Struggle_And_Lessons_For_Global_Security.pdf
biswajitbanerjee38
Ad

Plone on RelStorage

  • 1. Plone on RelStorage Calvin Hendryx-Parker PLONE CONFERENCE 2010 Saturday, October 30, 2010
  • 2. PLONE CONFERENCE 2010Overview How does Zope store data Why RelStorage? How to implement RelStorage Some reasons not to use RelStorage Saturday, October 30, 2010
  • 3. PLONE CONFERENCE 2010Zope Storage Methods Saturday, October 30, 2010
  • 4. PLONE CONFERENCE 2010What is the ZODB? Object oriented database Used by Zope to hold everything from content to scripts, to service con鍖gurations Saturday, October 30, 2010
  • 5. PLONE CONFERENCE 2010What is Filestorage? The default backend for Zope/ZEO var/鍖lestorage/Data.fs Does an in-memory index at startup Data appends to the end Back-up utility is Repozo Saturday, October 30, 2010
  • 6. PLONE CONFERENCE 2010What is RelStorage? Drop in replacement Replaces ZEO Eliminates Data.fs Stores pickles Is still ZODB Uses relational databases Saturday, October 30, 2010
  • 7. PLONE CONFERENCE 2010Why Change? Saturday, October 30, 2010
  • 8. PLONE CONFERENCE 2010Advantages of RelStorage Existing infrastructure Existing expertise Wide range of management tools High availability, high performance clusters No in-memory index at startup Scales well as your database grows High Concurrency Saturday, October 30, 2010
  • 9. PLONE CONFERENCE 2010Real World Major Health System Site Java portion of site locked up Restarted Serivce with Little Impact Saturday, October 30, 2010
  • 10. PLONE CONFERENCE 2010Databases Supported MySQL 5.0.32 or better MySQLdb >= 1.2.2 PostgreSQL 8.1 or better psycopg2 Oracle 10g cx_Oracle Saturday, October 30, 2010
  • 11. PLONE CONFERENCE 2010How Do I Install It? For MySQL add the following to buildout.cfg: eggs = RelStorage MySQL-Python fake-eggs = ZODB3 = 3.7.1 [instance] type mysql db plone user plone host localhost password terriblepassword Saturday, October 30, 2010
  • 12. PLONE CONFERENCE 2010How Do I Install It? For PostgreSQL add the following to buildout.cfg: eggs = RelStorage fake-eggs = ZODB3 = 3.7.1 psycopg2 [instance] type postgresql dbname plone user plone host localhost password terriblepassword Saturday, October 30, 2010
  • 13. PLONE CONFERENCE 2010Options blob-dir shared-blob-dir keep-history replica-conf pack-gc pack-dry-run cache-servers Saturday, October 30, 2010
  • 14. PLONE CONFERENCE 2010Available Tools zodbconvert Converts from 鍖lestorage to {mysql/pgsql/oracle} Convert from {mysql/pgsql/oracle} to 鍖lestorage Convert from one Database to another (ie: mysql to oracle) zodbpack Allows concurrent connections Use from cron Saturday, October 30, 2010
  • 15. PLONE CONFERENCE 2010zodbconvert.conf <filestorage source> path /opt/default/zinstance/var/ filestorage/Data.fs </filestorage> <relstorage destination> <mysql> db plone user plone passwd terriblepassword </mysql> </relstorage> Converting from 鍖lestorage to MySQL Saturday, October 30, 2010
  • 16. PLONE CONFERENCE 2010Running zodbconvert $ sudo /opt/mysql/zinstance/bin/zodbconvert --clear zodbconvert.conf Storages opened successfully. All transactions copied successfully. Saturday, October 30, 2010
  • 17. PLONE CONFERENCE 2010Whats the Catch? Saturday, October 30, 2010
  • 18. PLONE CONFERENCE 2010There Are Pitfalls Added complexity Additional server load Additional memory utilization The database is full of pickles, not plain text Your database still needs to be packed Saturday, October 30, 2010
  • 19. PLONE CONFERENCE 2010What We Talked About Zope storage engines: ZEO and RelStorage Why you might choose RelStorage How to install RelStorage Tools for RelStorage Why you might not use RelStorage Saturday, October 30, 2010
  • 20. PLONE CONFERENCE 2010Links http://pypi.python.org/pypi/RelStorage http://shane.willowrise.com/archives/relstorage-1-4-0b1-and-zodbshootout Saturday, October 30, 2010
  • 21. PLONE CONFERENCE 2010Credits: http://www.鍖ickr.com/photos/kaplanbr/ 2465380377/ http://www.鍖ickr.com/photos/ shuttercat7/416008944/ http://www.鍖ickr.com/photos/juniorvelo/ 3267647833/ http://www.鍖ickr.com/photos/ arthurjohnpicton/4383221264/ http://www.鍖ickr.com/photos/generated/ 3249283352/ Saturday, October 30, 2010