�ݺ�ߣ

A Robust Open-source GEDCOM Parser Dallan Quass [email_address] Ryan Knight [email_address]

What's a GEDCOM? 0 HEAD 1 SOUR PAF 2 NAME Personal Ancestral File 2 VERS 5.2.18.0 2 CORP The Church of Jesus Christ of Latter-day Saints 3 ADDR 50 East North Temple Street 4 CONT Salt Lake City, UT 84150 4 CONT USA 1 DEST Other 1 DATE 9 Aug 2006 2 TIME 19:57:47 1 FILE temp-paf.ged 1 GEDC 2 VERS 5.5 2 FORM LINEAGE-LINKED 1 CHAR UTF-8 1 LANG English 1 SUBM @SUB1@ 0 @SUB1@ SUBM 1 NAME Dallan Quass 0 @I1@ INDI 1 NAME Dallan /Quass/ 2 SURN Quass 2 GIVN Dallan If this looks unfamiliar to you, you may not get a lot out of this talk On the other hand, the purpose of this project is to handle this for you, so you can develop cool projects in genealogy and let this be unfamiliar to you!

Why is parsing GEDCOMs so hard?

Challenge #1 – Character set detection 0 HEAD 1 SOUR PAF 2 NAME Personal Ancestral File 2 VERS 5.2.18.0 2 CORP The Church of Jesus Christ of Latter-day Saints 3 ADDR 50 East North Temple Street 4 CONT Salt Lake City, UT 84150 4 CONT USA 1 DEST Other 1 DATE 9 Aug 2006 2 TIME 19:57:47 1 FILE temp-paf.ged 1 GEDC 2 VERS 5.5 2 FORM LINEAGE-LINKED 1 CHAR UTF-8 1 LANG English 1 SUBM @SUB1@ 0 @SUB1@ SUBM 1 NAME Dallan Quass 0 @I1@ INDI 1 NAME Dallan /Quass/ 2 SURN Quass 2 GIVN Dallan Should be easy, except...

Challenge #1 – Character set detection GeneWeb ASCII -> ANSI Geni.com ANSEL -> UTF8 Geni.com UNICODE -> UTF8 GENJ UNICODE -> UTF8 All others UNICODE -> UTF16 ASCII/MacOS Roman -> x-MacRoman

Challenge #1 – Character set detection ANSEL

Challenge #2 – Custom tags The GEDCOM specification hasn't been updated in a LONG time

Shout out Tim Forsythe VGed - GEDCOM validator http://ancestorsnow.blogspot.com/ 2011/07/vged.html

ALIA 1 SEX M 1 ALIA /Ted/ 1 BIRT

SOUR 0 @N6@ NOTE 1 CONT adopted surname Termaat 2 SOUR @S9@

DATA 2 SOUR @S2149874917@ 3 DATA 4 DATE 11 Sep 1924 3 NOTE ... 3 DATA 4 TEXT ... 2 SOUR @S99@ 3 DATA 4 TEXT William Donald ... 4 DATE 1 Sep 1997 2 SOUR @S28@ 3 PAGE Indian Prarie... 3 QUAY 3 3 DATE 28 Feb 2005

Challenge #4 – Unused tags Event Phone Event Agency Source Citation Event Type

GEDCOM Standard ? The code is more what you'd call " guidelines " than actual rules .

Goal #1 – Parse GEDCOMs into a de facto object model De Facto: In fact or in practice; in actual use or existence, regardless of official or legal status. – Wictionary.org Model should be straightforward, easy to use and understand

Goal #2 – Round-trip From GEDCOM To Object Model Back to GEDCOM without information loss

But we can get pretty close 94%

GedML Originally by Michael Kay http://users.breathe.com/mhkay/gedml/ Enhanced by Lynn Monson http://lmonson.com/blog/?page_id=64 Further enhanced by Nathan Powell & Dallan Quass part of this project GEDCOM -> SAX events ANSEL reader & writer

Parser Written in Java ~1500 LoC for parser + ~4000 LoC for POJOs Handles SAX events emitted by GedML Separate functions called to handle each tag Maintains a stack of model objects Attach unexpected tags to model objects as extensions Fast Easily extendible Tree parser also available

GEDCOM Export Visitor pattern 600 LoC

JSON GEDCOM POJO JSON POJO GEDCOM Simple model persistence using Google GSON

Do we need a radically-different data-exchange model for genealogy?

I don't know A new proposed object model could use this project to migrate existing GEDCOMs to the de facto model, then translate the de facto model objects to the new model

Do we need GEDCOM validation tools?

Definitely! A list of “standard” custom tags would also be pretty helpful

Demonstration of Gedcom Server Demonstrates GEDCOM -> model -> json -> model -> GEDCOM Built with Play 1.2.4 - A Java Web framework Allows for rapid development of web applications with a fully integrated stack Deployed to Heroku – Cloud Application Platform Heroku allows one step deployment with git

Demonstration of Gedcom Server

Conclusion Images appearing on these slides are copyrighted by the contributors to http://commons.wikimedia.org and are used under license Parsing GEDCOMs is hard it's like parsing HTML in the 1990's But getting it right is pretty important especially if you want to retain existing information Open source algorithm is now freely available http://github.com/DallanQ/Gedcom simple object model with extensions, 94% round-trip Hopefully others will benefit from this effort

�ݺ�ߣ

A Robust Open-source GEDCOM Parser

Recommended

More Related Content

What's hot (19)

Viewers also liked (19)

Similar to A Robust Open-source GEDCOM Parser (20)

A Robust Open-source GEDCOM Parser