狠狠撸

Transforming data silos into knowledge:
Early Chinese Periodicals Online (ECPO)
Matthias Arnold, Lena Hessel | Heidelberg | E-Science-Tage 2019 | 2019-03-29

Research data – Chinese periodical press
? First decades of the 20th century
? Understudied, but dominated the contemporary print market and
provide access to the "actual culture“ (R. Williams, 1961)
? Challenges:
? Physically dispersed, often poorly preserved
? Voluminous (full runs, daily, up to >30 years)
? Multi-generic and intellectually demanding
? Approach
? Multi-disciplinary team, >10 researchers
? Women and the Periodical Press in China’s Global
Twentieth Century: A Space of Their Own? Ed. by Joan
Judge, Barbara Mittler and Michel Hockx, Cambridge
University Press, 2018.
? Database
Early Chinese Periodicals Online (ECPO)

https://uni-heidelberg.de/ecpo

Transforming data silos into knowledge: Early Chinese Periodicals Online (ECPO)

276 publications: 134 with items

40.936 issues: 46.931 articles, 20.532 images, 18.639 ads

Chart: Publication activity by year
Arnold and Hessel | ECPO Database

Opening the data silo
From static export to dynamic data service
? Output data using the Metadata Object Description Schema
(MODS) - Open Access: http://ecpo.uni-hd.de/api/mods/
From static pre-rendered files to dynamic image service
? Implementation of International Image Interoperability
Framework (IIIF) Image API http://iiif.io/technical-details/
From separate names to cross-db agents service
? Identify agent, assign names, link to authorities, structure
information, feed data back to authority files (GND)

Agents Service

47.245 agents, 163.408 occurrences, 15 ‘languages’

Arnold, Heidelberg | Early Chinese Periodicals Online (ECPO): From Digitization to Open Data | JADH 2018

Arnold, Heidelberg | Early Chinese Periodicals Online (ECPO): Transforming data silos into knowledge | E-Science 2019
VIAF

VIAF
GND

VIAF
GND
Wikidata
VIAF

VIAF
GND
Wikidata
VIAF
Baidu baike

VIAF
GND
Wikidata
VIAF
Baidu baike
Agents with references to authorities:
VIAF: 861
Wikidata: 821
GND: 662
Baidu: 6
DBpedia: 5

Opening the Agents Service

Islington Corinthians F.C.:
- Leonard Bradbury
- Jack Braithwaite
- Alec Buchanan
- Pat Clark
- George Dance
- Cyril Longman
- Harry Lowe
- Richard Manning
- Albert (Eddie) Martin
- John Miller
- William Miller
- George Pearce
- Bert Read
- Johnny Sherwood
- Dick Tarrant
- Bill Whittaker
- Ted Wingfield
- J.K. Wright
Source: National Library Board
Singapore NewspaperSG,
accessed March 25, 2019,
http://eresources.nlb.gov.sg/new
spapers/Digitised/Article/straitsti
mes19371128-1.2.117.

Towards full text

https://uni-heidelberg.de/ecpo

Expanding data: towards fulltext
? Manual typing not feasible
? Professional double-keying very expensive
? OCR often unusable
? Document: dense layout, normal segmentation fails
? Image: noisy, secondary copies with stains/scratches
? Characters: special characters (emphasis), handwriting

Segmentation - I
? Page segmentation (pattern recognition/computer vision)
? Analyze layout of page, use page-internal structures
? Identify semantic units
? Generate co-ordinates, relate them to items, store in DB

Segmentation - II
? Page segmentation (crowdsourcing)
? Pilot project with Pallas Ludens GmbH
? Let the crowd help analyzing the pages
? Identify and label four item types:
? image/drawing
? article
? advertisement
? additional information
? Supervised
? Non-Chinese speaking community!

Processing
2. Page segmentation (computer vision/ocr)

Grouping semantic units
2. Page segmentation (crowdsourcing)
? drawing – correcting – grouping

Outcome of segmentation pilot
1. Page segmentation can be outsourced to expert crowd
? Requires supervision
? Advanced user interfaces (high usability, efficiency)
? Crowd should read Chinese (semantic grouping)
2. Jingbao 晶報 1919-21 completely segmented with qualified
boxes, issues of April 1919 with semantic units
3. Further processing:
? Partnership with Computational Knowledge Lab (知識計
算實驗室), Department of Engineering Science and
Ocean Engineering, Taiwan National University,
http://www.cklab.org/
? Seeking additional partners for collaboration!

Chinese Republican Periodicals –
Encoding full text in TEI

Mark-up: Different character sizes
<tagsDecl>
<rendition scheme="css" selector="body p">font-size:
100%;</rendition>
<rendition xml:id="half">font-size: 50%</rendition>
<rendition xml:id="double">font-size: 200%</rendition>
</tagsDecl>
<hi rendition="#double">女子之於男子</hi>
<hi rendition="#half">試觀西歐各國<lb/>名為男…

Mark-up: Emphasis
In Japanese: “emphasis dots” 圏点 (kenten) or 傍点
1. ? U+25E6 “open dot”
2. ? U+2022 “filled dot”
3. ○ U+25CB “open circle”
4. ● U+25CF “filled circle”
5. ◎ U+25CE “open double-circle”
6. ? U+25C9 “filled double-circle”
7. △ U+25B3 “open triangle”
8. ▲ U+25B2 “filled triangle”
9. ? U+FE46 “open sesame”
10. ? U+FE45 “filled sesame”
https://drafts.csswg.org/css-text-decor-3/#text-emphasis-style-property
BUT: emphasis characters mixed with
punctuation, differentiation and exact recording is
HUGE workload
-> emphasis characters currently ignored

Mark-up: Spaces between some characters
<space unit="chars" n="1"/>
OR
<gap unit="char" extent="1"> </gap>
(with “ ” being U+3000)
OR
just use U+3000 without markup

Wrap-up

From data silo towards open data
? Data collection = research data
? Enhance metadata
? Publishing information, content analysis (keywords)
? Separation of meta-/data from user interface
? FAIR Prinzipien
? DOI records for publications (in progress), connect database
to library catalogs
? Publish material and metadata Open Access, images,
publication metadata, and item metadata (article, image, ad)
? Basic data API (MODS XML)
open up IIIF manifests and Agents data (planned)
? Publish metadata on heiDATA/Dataverse (Summer)

Wrap-up
? Provide different ways to access data via frontend:
? Search (all metadata and annotations)
? Browse chronological (calendar)
? Browse/search agents / keywords
? Categories of publications
? Agents service (biographic data)
? cross-db record curation, connect persons with authorities
? plan (2019): add missing agents or names to GND, pull additional
data from authorities, develop agents API
? Page segmentation – crowdsourcing possible, grouping
requires Chinese, new tool creates web-annotations – seeking
partner for automatic page analysis
? Text – plan: process segments, generate full text, store TEI
XML, crowd-based editing

ECPO in a larger context
? Content expansion
? Early western publications printed in China
? Co-operation with Univ. Erlangen: Agents
? ECPO as data platform
? for storing, enhancing, accessing, sharing ?grey“
material from the CATS Library
? Outreach/ Communities
? DH-d working group Newspaper/Journals, OCR-d,
Transkribus/READ
? Connect with FID Asien (CrossAsia), Non-Latn scripts
interest group, TEI East Asia SIG
? Long-term repository: University Library, HeiDATA/HeidICON

Contact
Matthias Arnold – Lena Hessel
Heidelberg Centre for Transcultural Studies | HCTS
Karl Jaspers Centre
Vo?str. 2 | Building 4400 | Room 005b
69115 Heidelberg, Germany
Phone: +49 - 6221 - 54 4094
eMail: matthias.arnold@uni-hd.de
Web: http://tinyurl.com/matthias-arnold

狠狠撸

Transforming data silos into knowledge: Early Chinese Periodicals Online (ECPO)

More Related Content

Recently uploaded (20)

Featured (20)

Transforming data silos into knowledge: Early Chinese Periodicals Online (ECPO)