The document describes the creation of an Entity Summarization Benchmark (ESBM) to evaluate entity summarization systems. It outlines the design goals of ESBM, which include using multiple datasets, overcoming limitations of prior benchmarks, and being general-purpose. ESBM contains entity descriptions extracted from DBpedia and LinkedMDB, with over 6,500 triples and 2,100 human-generated summaries for 175 entities. Several entity summarization systems are evaluated using ESBM, demonstrating it is a challenging but useful benchmark for advancing research.
Building AI Applications using Knowledge GraphsAndre Freitas
油
This document provides an overview of building AI applications using knowledge graphs. It discusses the goals of the tutorial, which are to provide a broad view of multiple perspectives on knowledge graphs and show how knowledge graphs can form the foundation for building AI systems. The tutorial focuses on contemporary and emerging perspectives through exemplar approaches and infrastructures, rather than providing an exhaustive survey. It also notes that the tutorial is not a standard academic tutorial and takes a big picture view rather than being a comprehensive survey.
Combining Similarities and Regression for Entity Linking.C辿sar de Pablo
油
The document summarizes previous work on entity linking and knowledge base population tasks. It discusses the tasks of entity linking, which grounds entity mentions in documents to entries in a knowledge base, and slot filling, which learns attributes about target entities. It provides results from the TAC-KBP 2010 evaluation, showing entity linking accuracy for different entity types and domains. GPE entities were particularly difficult. Name similarity features and handling NIL queries impacted performance.
Grammatical Error Correction with Improved Real-world ApplicabilityMasato Mita
油
The document discusses grammatical error correction with improved real-world applicability. It discusses three main issues with current grammatical error correction:
[1] Evaluation - Current single corpus evaluations are not reliable and do not reflect real-world scenarios with varying proficiency levels.
[2] Data noise - There is noise in existing grammatical error correction training data that can negatively impact model performance.
[3] Low resource - Current approaches require large amounts of data and model parameters, which has low real-world applicability.
The document proposes approaches to address these issues, including cross-sectional evaluation using multiple test sets, a self-refinement strategy to reduce data noise, and analyzing grammatical generalization
Entity Summarization with User Feedback (ESWC 2020)Qingxia Liu
油
DRESSED is a method for entity summarization that incorporates user feedback to iteratively improve summaries. It models the interaction between the summarizer and user as a Markov decision process. DRESSED represents triple interdependence using a policy network that encodes triples and their relationships. It learns a policy to select replacement triples using REINFORCE policy gradient reinforcement learning. Experiments show DRESSED outperforms baselines in generating higher quality summaries over multiple iterations with user feedback.
Professor Steve Roberts; The Bayesian Crowd: scalable information combinati...Ian Morgan
油
Professor Steve Roberts, Machine learning research group and Oxford-Man Institute + Alan Turing Institute. Steve gave this talk on the 24th January at the London Bayes Nets meetup.
Semantic Search and Result Presentation with Entity CardsFaegheh Hasibi
油
The document describes semantic search and entity summarization techniques. It discusses generating query-dependent entity summaries using a knowledge graph. Facts about an entity are ranked based on importance and relevance to the query. The top facts are grouped by predicate and generated into a summary respecting length and width constraints. User studies found utility-based summaries outperformed relevance-only summaries. The techniques can power applications like knowledge-enabled search engines and personalized summaries.
Employees, Business Partners and Bad Guys: What Web Data Reveals About Person...Connotate
油
This document discusses how web data can reveal information about employees, business partners, and persons of interest. It outlines the business case for using web data to conduct background checks and screenings. It also discusses challenges like collecting good data from various sources and analyzing large amounts of unstructured data. Advanced text analytics solutions that use entity resolution and relationship extraction are presented as helping to understand web data. The document concludes by describing how these techniques were applied in a project with Thorn to detect child sex trafficking online.
This document summarizes a student project on sentiment analysis of online movie reviews. The student used movie review data from Kaggle and performed text preprocessing techniques like stemming and lemmatization. Bag of Words and TF-IDF models were used to represent the text data. Naive Bayes and Random Forest classifiers were applied and evaluated. TF-IDF with Naive Bayes achieved the best accuracy of 84.71%. The project involved common NLP tasks like data collection, preprocessing, modeling and evaluation.
A pilot on Semantic Textual Similaritypathsproject
油
This document summarizes the SemEval 2012 task on semantic textual similarity. It describes the motivation for the task as measuring similarity between text fragments on a graded scale. It then outlines the datasets used, including the MSR paraphrase corpus, MSR video corpus, WMT evaluation data, and OntoNotes word sense data. It also discusses the annotation process, which involved a pilot with authors and crowdsourcing through Mechanical Turk. The results showed most systems performed better than baselines and the best systems achieved correlations over 0.8 with human judgments.
H2O World - Intro to Data Science with Erin LedellSri Ambati
油
This document provides an introduction to data science. It defines data science as using data to solve problems through the scientific method. The roles of data scientists, data analysts, and data engineers on a data science team are discussed. Popular tools for data science include Python, R, and APIs that connect data processing engines. Machine learning algorithms are used to perform tasks like classification, regression, and clustering by learning from data rather than being explicitly programmed. Deep learning and ensemble methods are also introduced. Resources for learning more about data science and machine learning are provided.
Benchmarking graph databases on the problem of community detectionSymeon Papadopoulos
油
- The document presents a benchmark for evaluating the performance of graph databases Titan, OrientDB, and Neo4j on the task of community detection from graph data.
- OrientDB performed most efficiently for community detection workloads, while Titan was fastest for single insertion workloads and Neo4j generally had the best performance for querying and massive data insertion.
- Future work includes testing with larger graphs, running distributed versions of the databases, and improving the implemented community detection method.
This document discusses three case studies that use data analysis methods to address financial and risk-related questions. The first case study looks at predicting changes in corporate earnings using economic indicators. The second predicts the accuracy of Zillow home valuation estimates. The third examines factors that influence returns on initial public offerings of Japanese companies. The document then discusses dimensions of information quality that can impact the ability of a given dataset and analysis method to achieve a specified goal.
Search Accuracy Metrics and Predictive Analytics - A Big Data Use Case: Prese...Lucidworks
油
This document describes a new approach to evaluating search engine accuracy using predictive analytics and big data. The key points are:
- It presents a method to reliably measure and compare search engine accuracy offline using query logs and click logs, without requiring deployment to production.
- It analyzes activity at the user and session level to understand individual search behaviors and calculate engine scores based on relevance to each user.
- Leveraging big data, it uses a statistical model trained on past query and click data to predict the probability of relevance for new results, providing a more objective scoring method.
- This predictive relevance scoring approach identifies important parameters and allows experimenting to continuously improve search engine performance over time based on data and science
Benchmarking graph databases on the problem of community detectionSotiris Beis
油
- The document presents a benchmark for evaluating the performance of graph databases on the task of community detection from social networks. It tests Titan, OrientDB, and Neo4j on synthetic and real-world datasets.
- The results show that OrientDB is most efficient for community detection workloads, Titan performs best for single insertions, and Neo4j scales best for bulk insertions and queries.
- Future work includes testing with larger graphs, distributed versions of the databases, and improving the community detection method.
Co-clustering of multi-view datasets: a parallelizable approachAllen Wu
油
This document summarizes a research paper on co-clustering multi-view datasets using a parallelizable approach called MVSIM. MVSIM computes co-similarity matrices for related objects across multiple views or relation matrices. It creates a learning network matching the relational structure and aggregates the similarity matrices using a damping factor. Experiments show MVSIM outperforms single-view and other multi-view clustering methods on document and newsgroup datasets, and its performance decreases slightly but computation time reduces significantly when the data is split across more views.
Planning tools and techniques managementfizza tanvir
油
This document provides an overview of various planning tools and techniques discussed in Chapter 8 of a management textbook. It describes environmental scanning, forecasting, benchmarking, and other methods for assessing the external environment and allocating internal resources. These include quantitative and qualitative forecasting, steps in benchmarking, types of budgets, scheduling charts, break-even analysis, and linear programming. Contemporary techniques like project management and scenario planning are also summarized.
Planning tool and technique \\ Principle of managmentmmuhammadzulfqar5
油
This chapter discusses various planning tools and techniques for managers. It covers assessing the environment through techniques like environmental scanning, competitor intelligence, and forecasting. It then discusses allocating resources through budgeting, scheduling, Gantt and load charts, PERT network analysis, breakeven analysis, and linear programming. Finally, it covers contemporary planning techniques like project management, scenario planning, and contingency planning.
This chapter discusses various planning tools and techniques for managers. It covers assessing the environment through techniques like environmental scanning, competitor intelligence, and forecasting. It then discusses allocating resources through budgeting, scheduling, Gantt and load charts, PERT network analysis, breakeven analysis, and linear programming. Finally, it covers contemporary planning techniques like project management, scenario planning, and contingency planning.
Knowledge Graph Curation: A Practical FrameworkElwin Huaman
油
Knowledge Graphs (KGs) are very important for applications such as personal assistants, question-answering systems, and search engines. However, KGs inevitably contain wrong assertions, duplicates, or missing values, i.e., low-quality KGs produce low-quality applications that are built on top of them. Therefore, we propose a KG Curation Framework, which involves the assessment, cleaning, and enrichment of KGs.
The document provides a project implementation report analyzing Net Promoter Score (NPS) for Hyatt Hotel Group. Key analyses included identifying patterns in customer visits on Valentine's Day across the US, determining which guest demographics are most likely to be promoters, and correlations between nightly rates, length of stay, purpose of visit, and customer satisfaction. Recommendations include offering Valentine's packages, discounts for older and younger guests, and adjusting nightly rates based on length of stay patterns in different countries. The report covered results from a variety of statistical methods to analyze the hotel customer data and provide actionable insights.
The document discusses developing an opinion-driven decision support system (ODSS). It proposes that an ODSS should have four main components: 1) a comprehensive set of opinion data, 2) tools for analyzing and digesting opinions, 3) capabilities for searching for entities based on opinions, and 4) effective presentation of opinions to support decision making. Currently, most work focuses on opinion summarization and structured summaries. However, an ODSS requires addressing broader problems like enabling opinion-based search and developing different analysis tools to help users make decisions based on opinions.
This document provides an overview of foundational research propelled by text analytics. It begins with an outline that discusses text analytics in the big data era, information extraction systems and formalisms, foundational research challenges, and conclusions. It then discusses how text analytics has become important for applications like semantic search, life science mining, e-commerce, CRM/BI, and log analysis. It notes the need for database management systems and general-purpose development and management systems to facilitate value extraction from big data by a wide range of users and skills. Core information extraction tasks like named entity recognition, relation extraction, event extraction, temporal information extraction, and coreference resolution are discussed. Several formalisms for information extraction are presented, including X
Comparison Study of Neural Network and Deep Neural Network on Repricing GAP P...Hendri Karisma
油
This document summarizes a study that compared neural network and deep neural network models for predicting repricing gaps in Indonesian banks. The study used monthly report data from 2003-2013 to construct datasets for evaluating the models. Deep neural networks had better performance than standard backpropagation neural networks, achieving lower error rates with faster convergence. The deep learning approach was able to better handle the nonlinear and missing data characteristics of the bank reports. The researchers concluded deep neural networks are a promising approach for repricing gap prediction on Indonesian bank data.
Here are some science-related events from EventKG that took place in Lyon:
- 1921: " Lyon, fusion de la Soci辿t辿 de m辿decine et de la Soci辿t辿 des sciences m辿dicales" (In Lyon, merger of the Medical Society and the Society of Medical Sciences)
- 1987: "The International Astronomical Union organizes its 24th General Assembly in Lyon"
- 1988: "The International Astronomical Union organizes its 25th General Assembly in Lyon"
- 2009: "The International Astronomical Union organizes its 26th General Assembly in Lyon"
- 2015: "The International Astronomical Union organizes its 29th General Assembly in Lyon"
-
Benchmarking Automated Machine Learning For Clusteringbiagiolicari7
油
This document discusses benchmarking four automated machine learning (AutoML) frameworks for clustering: AutoML4Clust, cSmartML, Autocluster, and ML2DAC. It describes the benchmark design, evaluation criteria of clustering quality, scalability, and consistency. The results show that ML2DAC emerged as the top performer based on clustering validity indices and Bayesian analysis, though it was not consistently the best. Room remains for improving AutoML frameworks' performance and transparency for clustering tasks.
The document discusses data workflows and integrating open data from different sources. It defines a data workflow as a series of well-defined functional units where data is streamed between activities such as extraction, transformation, and delivery. The document outlines key steps in data workflows including extraction, integration, aggregation, and validation. It also discusses challenges around finding rules and ontologies, data quality, and maintaining workflows over time. Finally, it provides examples of data integration systems and relationships between global and source schemas.
This document summarizes a student project on sentiment analysis of online movie reviews. The student used movie review data from Kaggle and performed text preprocessing techniques like stemming and lemmatization. Bag of Words and TF-IDF models were used to represent the text data. Naive Bayes and Random Forest classifiers were applied and evaluated. TF-IDF with Naive Bayes achieved the best accuracy of 84.71%. The project involved common NLP tasks like data collection, preprocessing, modeling and evaluation.
A pilot on Semantic Textual Similaritypathsproject
油
This document summarizes the SemEval 2012 task on semantic textual similarity. It describes the motivation for the task as measuring similarity between text fragments on a graded scale. It then outlines the datasets used, including the MSR paraphrase corpus, MSR video corpus, WMT evaluation data, and OntoNotes word sense data. It also discusses the annotation process, which involved a pilot with authors and crowdsourcing through Mechanical Turk. The results showed most systems performed better than baselines and the best systems achieved correlations over 0.8 with human judgments.
H2O World - Intro to Data Science with Erin LedellSri Ambati
油
This document provides an introduction to data science. It defines data science as using data to solve problems through the scientific method. The roles of data scientists, data analysts, and data engineers on a data science team are discussed. Popular tools for data science include Python, R, and APIs that connect data processing engines. Machine learning algorithms are used to perform tasks like classification, regression, and clustering by learning from data rather than being explicitly programmed. Deep learning and ensemble methods are also introduced. Resources for learning more about data science and machine learning are provided.
Benchmarking graph databases on the problem of community detectionSymeon Papadopoulos
油
- The document presents a benchmark for evaluating the performance of graph databases Titan, OrientDB, and Neo4j on the task of community detection from graph data.
- OrientDB performed most efficiently for community detection workloads, while Titan was fastest for single insertion workloads and Neo4j generally had the best performance for querying and massive data insertion.
- Future work includes testing with larger graphs, running distributed versions of the databases, and improving the implemented community detection method.
This document discusses three case studies that use data analysis methods to address financial and risk-related questions. The first case study looks at predicting changes in corporate earnings using economic indicators. The second predicts the accuracy of Zillow home valuation estimates. The third examines factors that influence returns on initial public offerings of Japanese companies. The document then discusses dimensions of information quality that can impact the ability of a given dataset and analysis method to achieve a specified goal.
Search Accuracy Metrics and Predictive Analytics - A Big Data Use Case: Prese...Lucidworks
油
This document describes a new approach to evaluating search engine accuracy using predictive analytics and big data. The key points are:
- It presents a method to reliably measure and compare search engine accuracy offline using query logs and click logs, without requiring deployment to production.
- It analyzes activity at the user and session level to understand individual search behaviors and calculate engine scores based on relevance to each user.
- Leveraging big data, it uses a statistical model trained on past query and click data to predict the probability of relevance for new results, providing a more objective scoring method.
- This predictive relevance scoring approach identifies important parameters and allows experimenting to continuously improve search engine performance over time based on data and science
Benchmarking graph databases on the problem of community detectionSotiris Beis
油
- The document presents a benchmark for evaluating the performance of graph databases on the task of community detection from social networks. It tests Titan, OrientDB, and Neo4j on synthetic and real-world datasets.
- The results show that OrientDB is most efficient for community detection workloads, Titan performs best for single insertions, and Neo4j scales best for bulk insertions and queries.
- Future work includes testing with larger graphs, distributed versions of the databases, and improving the community detection method.
Co-clustering of multi-view datasets: a parallelizable approachAllen Wu
油
This document summarizes a research paper on co-clustering multi-view datasets using a parallelizable approach called MVSIM. MVSIM computes co-similarity matrices for related objects across multiple views or relation matrices. It creates a learning network matching the relational structure and aggregates the similarity matrices using a damping factor. Experiments show MVSIM outperforms single-view and other multi-view clustering methods on document and newsgroup datasets, and its performance decreases slightly but computation time reduces significantly when the data is split across more views.
Planning tools and techniques managementfizza tanvir
油
This document provides an overview of various planning tools and techniques discussed in Chapter 8 of a management textbook. It describes environmental scanning, forecasting, benchmarking, and other methods for assessing the external environment and allocating internal resources. These include quantitative and qualitative forecasting, steps in benchmarking, types of budgets, scheduling charts, break-even analysis, and linear programming. Contemporary techniques like project management and scenario planning are also summarized.
Planning tool and technique \\ Principle of managmentmmuhammadzulfqar5
油
This chapter discusses various planning tools and techniques for managers. It covers assessing the environment through techniques like environmental scanning, competitor intelligence, and forecasting. It then discusses allocating resources through budgeting, scheduling, Gantt and load charts, PERT network analysis, breakeven analysis, and linear programming. Finally, it covers contemporary planning techniques like project management, scenario planning, and contingency planning.
This chapter discusses various planning tools and techniques for managers. It covers assessing the environment through techniques like environmental scanning, competitor intelligence, and forecasting. It then discusses allocating resources through budgeting, scheduling, Gantt and load charts, PERT network analysis, breakeven analysis, and linear programming. Finally, it covers contemporary planning techniques like project management, scenario planning, and contingency planning.
Knowledge Graph Curation: A Practical FrameworkElwin Huaman
油
Knowledge Graphs (KGs) are very important for applications such as personal assistants, question-answering systems, and search engines. However, KGs inevitably contain wrong assertions, duplicates, or missing values, i.e., low-quality KGs produce low-quality applications that are built on top of them. Therefore, we propose a KG Curation Framework, which involves the assessment, cleaning, and enrichment of KGs.
The document provides a project implementation report analyzing Net Promoter Score (NPS) for Hyatt Hotel Group. Key analyses included identifying patterns in customer visits on Valentine's Day across the US, determining which guest demographics are most likely to be promoters, and correlations between nightly rates, length of stay, purpose of visit, and customer satisfaction. Recommendations include offering Valentine's packages, discounts for older and younger guests, and adjusting nightly rates based on length of stay patterns in different countries. The report covered results from a variety of statistical methods to analyze the hotel customer data and provide actionable insights.
The document discusses developing an opinion-driven decision support system (ODSS). It proposes that an ODSS should have four main components: 1) a comprehensive set of opinion data, 2) tools for analyzing and digesting opinions, 3) capabilities for searching for entities based on opinions, and 4) effective presentation of opinions to support decision making. Currently, most work focuses on opinion summarization and structured summaries. However, an ODSS requires addressing broader problems like enabling opinion-based search and developing different analysis tools to help users make decisions based on opinions.
This document provides an overview of foundational research propelled by text analytics. It begins with an outline that discusses text analytics in the big data era, information extraction systems and formalisms, foundational research challenges, and conclusions. It then discusses how text analytics has become important for applications like semantic search, life science mining, e-commerce, CRM/BI, and log analysis. It notes the need for database management systems and general-purpose development and management systems to facilitate value extraction from big data by a wide range of users and skills. Core information extraction tasks like named entity recognition, relation extraction, event extraction, temporal information extraction, and coreference resolution are discussed. Several formalisms for information extraction are presented, including X
Comparison Study of Neural Network and Deep Neural Network on Repricing GAP P...Hendri Karisma
油
This document summarizes a study that compared neural network and deep neural network models for predicting repricing gaps in Indonesian banks. The study used monthly report data from 2003-2013 to construct datasets for evaluating the models. Deep neural networks had better performance than standard backpropagation neural networks, achieving lower error rates with faster convergence. The deep learning approach was able to better handle the nonlinear and missing data characteristics of the bank reports. The researchers concluded deep neural networks are a promising approach for repricing gap prediction on Indonesian bank data.
Here are some science-related events from EventKG that took place in Lyon:
- 1921: " Lyon, fusion de la Soci辿t辿 de m辿decine et de la Soci辿t辿 des sciences m辿dicales" (In Lyon, merger of the Medical Society and the Society of Medical Sciences)
- 1987: "The International Astronomical Union organizes its 24th General Assembly in Lyon"
- 1988: "The International Astronomical Union organizes its 25th General Assembly in Lyon"
- 2009: "The International Astronomical Union organizes its 26th General Assembly in Lyon"
- 2015: "The International Astronomical Union organizes its 29th General Assembly in Lyon"
-
Benchmarking Automated Machine Learning For Clusteringbiagiolicari7
油
This document discusses benchmarking four automated machine learning (AutoML) frameworks for clustering: AutoML4Clust, cSmartML, Autocluster, and ML2DAC. It describes the benchmark design, evaluation criteria of clustering quality, scalability, and consistency. The results show that ML2DAC emerged as the top performer based on clustering validity indices and Bayesian analysis, though it was not consistently the best. Room remains for improving AutoML frameworks' performance and transparency for clustering tasks.
The document discusses data workflows and integrating open data from different sources. It defines a data workflow as a series of well-defined functional units where data is streamed between activities such as extraction, transformation, and delivery. The document outlines key steps in data workflows including extraction, integration, aggregation, and validation. It also discusses challenges around finding rules and ontologies, data quality, and maintaining workflows over time. Finally, it provides examples of data integration systems and relationships between global and source schemas.
Explore Mobile, Alabama's population growth, trends, and key insights. Discover how the city's demographics, economy, and urban development have evolved over the years. Learn about population density, migration patterns, and future projections shaping Mobiles growth. Whether you're a resident, investor, or researcher, stay informed about Mobiles changing landscape and what it means for businesses, housing, and the local community. Get the latest updates on Mobile, Alabamas population trends today!
Play-it-Forward: A Sustainable Sports-Initiative in Zambiarenubharathi99
油
This presentation explores Play it Forward, a project to build a sustainable sports facility in Zambia. It focuses on community development, financial planning, funding strategies, and long-term sustainability. Key aspects include stakeholder engagement, income diversification, and strategic partnerships to ensure success.
Public Relations Research Presentation 2024al20303
油
ESBM: An Entity Summarization Benchmark (ESWC 2020)
1. ESBM: An Entity Summarization Benchmark
Qingxia Liu1, Gong Cheng1, Kalpa Gunaratna2, and Yuzhong Qu1
1 National Key Laboratory for Novel Software Technology, Nanjing University, China
2 Samsung Research America, Mountain View CA, USA
3. 2020.06 3
Entity Summarization
<Tim Berners Lee, alias, TimBL>
<Tim Berners Lee, name, Tim Berners-Lee>
<Tim Berners Lee, givenName, Tim>
<Tim Berners Lee, birthYear, 1955>
<Tim Berners Lee, birthDate, 1955-06-08>
<Tim Berners Lee, birthPlace, England>
<Tim Berners Lee, birthPlace, London>
<Tim Berners Lee, type, People Educated At Emanuel School>
<Tim Berners Lee, type, Scientist>
<Tim Berners-Lee, child, Ben Berners-Lee>
<Tim Berners-Lee, child, Alice Berners-Lee>
<Conway Berners-Lee, child, Tim Berners-Lee>
<Weaving the Web, author, Tim Berners-Lee>
<Tabulator, author, Tim Berners-Lee>
<Paul Otlet, influenced, Tim Berners-Lee>
<John Postel, influenced, Tim Berners-Lee>
<World Wide Web, developer, Tim Berners-Lee>
<World Wide Web Foundation, foundedBy, Tim Berners-Lee>
<World Wide Web Foundation, keyPerson, Tim Berners-Lee><Tim Berners Lee, type, Living People>
<Tim Berners Lee, type, Person>
<Tim Berners Lee, type, Agent>
<Tim Berners-Lee, award, Royal Society>
<Tim Berners-Lee, award, Royal Academy of Engineering >
<Tim Berners-Lee, award, Order of Merit>
<Tim Berners-Lee, award, Royal Order of the British Empire>
<Tim Berners-Lee, spouse, Rosemary Leith>
<Tim Berners Lee, birthDate, 1955-06-08>
<Tim Berners Lee, birthPlace, England>
<Tim Berners Lee, type, Scientist>
<Tim Berners-Lee, award, Royal Society>
<World Wide Web, developer, Tim Berners-Lee>
Description of Tim Berners-Lee:
Summary:
4. RDF Data: T
triple tT: <subj, pred, obj>
Entity Description: Desc(e)
Desc(e) ={tT: subj(t)=e or obj(t)=e}
triple tDesc(e): <e, property, value>
values: class, entity, literal
Entity Summarization (ES): S(e, k)
SDesc(e) , |S|k
2020.06 4
Entity Summarization
Tim Berners-Lee
England
Scientist
Royal Society
Weaving the Web
Person
Paul Otlet
Tim Tim Berners-Lee
John Postel
1955-06-08
1955
valuesproperties
birthPlace type
type
author
influenced
influenced
name
givenName birthYear
birthDate
award
5. Limitations
Task specificness
Single dataset
Small size
Triple incomprehensiveness
2020.06 5
Existing Benchmarks
1 http://yovisto.com/labs/iswc2012
2 http://wiki.knoesis.org/index.php/FACES
6. Motivation
Research Challenges for Entity Summarization:
Lack of good benchmarks
Lack of evaluation efforts
Contributions
Created an Entity Sumarization Benchmark (ESBM v1.2)
overcoming the limitations of existing benchmarks
meeting the desiderata for a successful benchmark
Evaluated entity summarizers with ESBM
made the most extensive evaluation effort to date
evaluated 9 existing general-purpose entity summarizers
evaluated 1 supervised learning-based entity summarizer for reference
2020.06 6
Our Work
8. To satisfy seven desiderata for a successful benchmark[18]
accessibility, affordability, clarity, relevance, solvability, portability, scalability
To overcome limitations of available benchmarks
General-purpose summaries
Including class-, entity-, literal-valued triples
Multiple datasets
Currently largest available benchmark
2020.06 8
Design Goals
[18] Sim, S.E., Easterbrook, S.M., Holt, R.C.: Using benchmarking to advance research: A challenge to software engineering. In: ICSE 2003. pp. 74{83 (2003).
9. Datasets
DBpedia
imported dump files: instance types, instance types transitive, YAGO types, mappingbased
literals, mappingbased objects, labels, images, homepages, persondata, geo coordinates
mappingbased, and article categories
LinkedMDB
removed triples: owl:sameAs
Entities
sampled from seven large classes:
DBpedia: Agent, Event, Location, Species, Work
LinkedMDB: Film, Person
Triples per entity
By class: 25.88-52.44 triples
Overall: 37.62 triples
2020.06 9
Entity Descriptions
10. 2020.06 10
Ground-Truth Summaries
Task
30 users
each assigned 35 entities
175 entities
each assigned to 6 users
Each user created two
summaries for each entity
for k=5 and k=10
Total
6 top-5 summaries
and 6 top-10 summaries
for each entity
175*6*2=2100 ground-truth summaries
11. Usage
ESBM v1.2: specified training-validation-test splits for 5-fold cross validation
Early versions: EYRE 2018 workshop, EYRE 2019 workshop
Desiderata
Accessibility: permanent identifier on w3id.org
Affordability: open-source, example code for evaluation
Clarity: documented clearly and concisely
Relevance: entities sampled from real datasets
Solvability: not trivial and not too difficult
Portability: any general-purpose entity summarizer that can process RDF data
Scalability: reasonably large and diverse to evaluate mature entity summarizers
2020.06 11
The ESBM Benchmark
13. 175 entities, 6584 triples, 2100 ground-truth summaries
2020.06 13
Basic Statistics
Proportion of triples been selected into ground-truth summaries
Overlap: 4.91 triples
Top-5
summary
Top-10
summary
Overlap between top-5 and top-10 summaries
14. Literal-valued triples constitute a large proportion in ground-truth summaries.
30% in top-5 ground-truth summaries and 25% in top-10 summaries
Participants are not inclined to select multiple values of a property.
The average number of distinct properties in top-5 ground-truth summaries is 4.70 (very close to 5)
2020.06 14
Triple Composition
Three bars in each group: Entity descriptions, Top-5 ground-truth summaries, Top-10 ground-truth summaries
15. Entity Description
Jaccard similarity between property sets from each pair of classes is very low.
2020.06 15
Entity Heterogeneity
16. Ground-truth Summaries
Popular properties:
properties that appear in >50% ground truth summaries for each class
Only 1~2/13.24 properties are popular in top-5 ground-truth summaries
The importance of properties is generally contextualized by concrete entities.
2020.06 16
Entity Heterogeneity
17. Average overlap between 6 ground-truth summaries
Moderate degree of agreement
Comparable with those reported for other benchmarks
2020.06 17
Inter-Rater Agreement
[2] Cheng, G., Tran, T., Qu, Y.: RELIN: relatedness and informativeness-based centrality for entity summarization. In: ISWC 2011, Part I. pp. 114-129 (2011).
[7] Gunaratna, K., Thirunarayan, K., Sheth, A.P.: FACES: diversity-aware entity summarization using incremental hierarchical conceptual clustering. In: AAAI 2015. pp. 116-122 (2015).
[8] Gunaratna, K., Thirunarayan, K., Sheth, A.P., Cheng, G.: Gleaning types for literals in RDF triples with application to entity summarization. In: ESWC 2016. pp. 85-100 (2016).
19. Existing Entity Summarizers
RELIN, DIVERSUM, LinkSUM, FACES, FACES-E, CD
MPSUM, BAFREC, KAFCA
ORACLE Entity Summarizer
k triples that are selected by the most participants into ground-truth summaries
Supervised Learning-Based Entity Summarizer
6 models:
SMOreg, LinearRegression, MultilayerPerceptron, AdditiveRegression, REPTree,
RandomForest
7 features:
gfT(global frequency of property), lf(local frequency of property), vfT(frequency of value),
si(self-information of triple)
isC(value is class), isE(value is entity), isL(value is literal)
2020.06 19
Participating Entity Summarizers
20. Evaluation Criteria
Sm: machine-generated entity summary
Sh : human-made ground-truth summary
PR if |Sm|<|Sh |=k
2020.06 20
Settings
23. F1 results
RandomForest, REPTree
achieve the highest F1.
Four methods
outperform all the
existing entity
summarizers.
Two methods only fail to
outperform existing
entity summarizers in one
setting.
2020.06 23
Results of Supervised Learning
Demonstrated the powerfulness of supervised learning for entity summarization.
24. 2020.06 24
Results of Supervised Learning
Features
for each t=<e, p,v> in Desc(e):
gfT: # triples in the dataset where p appears
lf: # triples in Desc(e) where p appears
vfT: # triples in dataset where v appears
si: self-information of triple t
isC: whether v is a class
isE: whether v is an entity
isL: whether v is a literal
Results
significantly effective: gfT, lf
for LinkedMDB: vfT, si
not significant: isC, isE, isL
25. Existing entity summarizers
Leading systems: BAFREC, MPSUM
Supervised Learning method
Outperforms existing entity summarizers
Comparing with ORACLE
Still a large gap for improvement
2020.06 25
Summary of Evaluation Results
Entity summarization on ESBM is a non-trivial task.
27. Evaluation Criteria
semantic overlap between triples
Representativeness of Ground Truth
general-purpose VS. task-specific
Form of Ground Truth
set-based VS. scoring-based
2020.06 27
Limitations
28. Contributions
Created an Entity Summarization Benchmark: ESBM
overcoming the limitations of existing benchmarks
Evaluated entity summarizers with ESBM
the most extensive evaluation effort to date
ESBM
The currently largest available benchmark for entity summarization
Entity summarization on ESBM is a non-trivial task
Permanent link: https://w3id.org/esbm/
GitHub repository: nju-websoft/ESBM
2020.06 28
Take-home Message
An Upcoming Paper
Junyou Li, Gong Cheng, Qingxia Liu, Wen Zhang, Evgeny Kharlamov, Kalpa Gunaratna, Huajun Chen.
Neural Entity Summarization with Joint Encoding and Weak Supervision.
IJCAI-PRICAI 2020
Deep learning based
Significantly outperformed all the existing systems on ESBM
30. Contributions
Created an Entity Summarization Benchmark: ESBM
overcoming the limitations of existing benchmarks
Evaluated entity summarizers with ESBM
the most extensive evaluation effort to date
ESBM
The currently largest available benchmark for entity summarization
Entity summarization on ESBM is a non-trivial task
Permanent link: https://w3id.org/esbm/
GitHub repository: nju-websoft/ESBM
2020.06 30
Take-home Message
An Upcoming Paper
Junyou Li, Gong Cheng, Qingxia Liu, Wen Zhang, Evgeny Kharlamov, Kalpa Gunaratna, Huajun Chen.
Neural Entity Summarization with Joint Encoding and Weak Supervision.
IJCAI-PRICAI 2020
Deep learning based
Significantly outperformed all the existing systems on ESBM