Python, web scraping and content management: Scrapy and DjangoSammy Fung
?
This document discusses using Python, Scrapy, and Django for web scraping and content management. It provides an overview of open data principles and describes how Scrapy can be used to extract structured data from websites. Scrapy spiders can be defined to scrape specific sites and output extracted data. Django is introduced as a web framework for building content management systems. The document demonstrates how Scrapy and Django can be integrated, with Scrapy scraping data and Django providing data models and administration. It also describes the hk0weather project on GitHub as an example that scrapes Hong Kong weather data using these tools.
Downloading the internet with Python + ScrapyErin Shellman
?
The document describes using the Python library Scrapy to build web scrapers and extract structured data from websites. It discusses monitoring competitor prices as a motivation for scraping. It provides an overview of Scrapy projects and components. It then walks through setting up a Scrapy spider to scrape product data from the backcountry.com website, including defining items to scrape, crawling and parsing instructions, requesting additional pages, and cleaning extracted data. The goal is to build a scraper that extracts product and pricing information from backcountry.com to monitor competitor prices.
Global Open Source Development 2011-2014 Review and 2015 ForecastSammy Fung
?
1) The document reviews global open source development from 2011-2014 and provides a forecast for 2015. Key technologies discussed include JavaScript, Node.js, Python, R, and open data.
2) Major open source organizations discussed include Mozilla, the Python Software Foundation, and the Open Knowledge Foundation. Events like PyCon and the Open Data Day hackathon are also mentioned.
3) The document concludes by discussing opportunities to get involved in and contribute to the global open source community, such as upcoming events in Hong Kong like HKOSCon 2015 and PyCon HK 2015.
How to scraping content from web for location-based mobile app.Diep Nguyen
?
This document discusses using web scraping to collect location-based data for mobile apps. It introduces web scraping and the Scrapy framework. It then proposes using Scrapy to continuously scrape data like addresses, latitude/longitude coordinates, and phone numbers from various websites. The scraped data would be extracted using techniques like XPath and geocoding APIs. Duplicated data would be prevented and the scraping system could run without a dedicated server by syncing scraped results to a server.
This document discusses the Gradle Play Publisher Plugin, which allows uploading an Android app and its details to the Google Play Store from a Gradle build. It presents the agenda, explains why the plugin is useful for automating the tedious manual upload process, describes how to set up the plugin by creating a Google service account and configuring Gradle, and demonstrates its tasks and directory structure. References for the Google Publishing API and plugin are provided at the end.
COSCUP 2013 was a 2-day Taiwanese conference held at the Taipei International Conference Centre with over 1,500 participants across 8 tracks in Mandarin and English. It featured keynote speeches from Greg Kroah-Hartman, the Linux kernel maintainer, and other speakers from Taiwan, Europe, America, Japan, Hong Kong, and China across communities like Drupal, JavaScript, Mozilla, Python, R, and more. Registration for open source contributors before May 31st sold out within 30 minutes.
Sammy Fung presented on his open source weather projects hk0weather and TCTrack, as well as his views on open data and open government in Hong Kong. He described hk0weather, an open source project to extract weather data from the Hong Kong Observatory website and convert it to JSON format. He discussed challenges with obtaining weather data from the HKO and limitations of weather data provided through the Hong Kong government's open data portal. He also proposed ideas for improving the transparency of legislative council vote result data.
DevRel - Transform article writing from printing to onlineSammy Fung
?
The document outlines the transformation of article writing from print to online focusing on developer relations (DevRel) and community engagement. It discusses collaborative interactions between companies and developers, community gatherings, conferences, and hackathons that foster business and technology activities. Additionally, it highlights the opportunities for new partnerships and ideas that emerge in the DevRel ecosystem.
Introduction to Open Source by opensource.hk (2019 Edition)Sammy Fung
?
OpenSource.hk is a community focused on promoting open source software in Hong Kong, supporting various contributions such as events and mentoring. It operates under a Creative Commons license and encourages collaboration and participation from volunteers. The document discusses open source software development processes, business models, and highlights various resources and educational materials related to open source.
My Open Source Journey - Developer and CommunitySammy Fung
?
Sammy Fung shares his journey in the open source community, highlighting his self-taught programming skills and contributions to various projects using languages like Python, Perl, and PHP. He emphasizes the value of open source software for personal and professional growth, detailing his roles in leading organizations like HKLUG and Open Source Hong Kong while navigating challenges in volunteer recruitment and community engagement. Fung advocates for balance between community service and personal income, urging others to support and contribute to the open source ecosystem.
Introduction to development with Django web frameworkSammy Fung
?
This document introduces Django web framework development. It discusses installing and setting up Django, creating a sample "eventorg" project with an "eventmgr" app, designing data models and accessing them, designing URLs and views, and designing templates. The goal is to teach the reader how to create a basic Django application from start to finish.
Sammy Fung discusses open data and web APIs in his presentation at Hong Kong Polytechnic University, emphasizing the importance of structured and machine-readable data. The presentation covers how to access data like air quality indexes using web scraping techniques and highlights the concept of open data deployments with a 5-star rating system. It also explains the role of APIs in data retrieval and touches upon the development and community around open source software.
Mozilla is a non-profit organization that promotes openness, innovation, and opportunity on the web through products like Firefox and Thunderbird. Its mission is to ensure the web remains open and accessible to all. Mozilla also develops technologies like Webmaker to teach people how to use and contribute to the open web. The presentation announces upcoming events in June 2015 on open source and Mozilla in Hong Kong to bring people together to learn and contribute to the open web.
This document summarizes Sammy Fung's involvement in open source technology and community over the past few decades. It notes that Sammy is a founding chairman of HKLUG, president of Open Source Hong Kong, and a member of GNOME Asia Committee. It then discusses Sammy's journey learning about and contributing to open source since 1991, including founding various open source organizations in Hong Kong. Finally, it encourages participating in open source communities to develop skills, networks, and have fun while contributing to society.
Access Open Data with Open Source Software ToolsSammy Fung
?
The document discusses using open data and open source software tools. It provides an overview of open data and open source, describing key concepts like freedoms for users. Popular open source stacks like LAMP are presented, along with tools like Python. The document outlines several of the author's past open data projects and describes a current project scraping Hong Kong restaurant license data using Scrapy, an open source web crawling framework. The project code is available on GitHub.
Installation of LAMP Server with Ubuntu 14.10 Server EditionSammy Fung
?
This document provides a comprehensive guide to installing a LAMP server using Ubuntu 14.10 server edition, detailing system requirements and alternatives. It covers configuration steps for Linux, Apache, MySQL, and PHP, alongside practical commands and tips for setting up a web server environment. Additionally, it includes insights on user management, networking, and database handling within the Linux operating system.
Software Freedom and Open Source CommunitySammy Fung
?
This document discusses open source software and the open source community. It describes key open source projects like Linux, Apache, Firefox, and their histories. It also outlines four essential freedoms of free software. Additionally, it discusses Hong Kong open source organizations and events that promote open source software, including OpenSource.HK and Hong Kong Creative Open Technology Association. Finally, it provides an overview of the Google Code-in and Google Summer of Code programs that introduce students to open source development.
This document discusses building a job site using the open source content management system Drupal. It describes how the speaker developed an open source Drupal module for job postings after encountering errors on an existing job site. The module allows users to post and categorize jobs and view category-specific RSS feeds. Instructions are provided on setting up a Linux web server and installing Drupal and relevant modules to build a customizable job board website.
Use open source software to develop ideas at workSammy Fung
?
The document discusses the importance of open source software in future market leadership, emphasizing the freedoms it provides to users and the community. It highlights the evolution of open source technologies, major contributions like Linux and Apache, and examples of open source projects such as a job board and weather data parser. The content also outlines opportunities for community involvement in open source through various organizations and events in Hong Kong.
Software Freedom Day 2014 in Hong Kong featured talks on free software and community, technology use, and contributions from various speakers. The event was organized by multiple groups including Open Source Hong Kong and the Hong Kong Linux User Group, with a focus on the four essential freedoms of free software. The document also highlights the development of key technologies such as the Linux operating system and Apache web server, alongside the importance of community engagement in open-source development.
Introduction of Mozilla Hong Kong (COSCUP 2014)Sammy Fung
?
Sammy Fung is a developer and Mozilla representative who serves as the Mozilla Hong Kong Community Liaison. He is the president of OpenSource.HK and director at HKCOTA. He participates in open source communities in Asia and works on projects like the zh_HK localization of Firefox and the Mozilla Hong Kong community website at mozilla.hk.
Introduction of Open Source Job Board with Drupal CMSSammy Fung
?
This document introduces an open source job board built with the Drupal content management system (CMS). It discusses how the job board allows for posting and categorizing jobs, company profiles, RSS feeds, and was developed using open source software. The job board code is available on GitHub and support services are offered to help organizations install, customize, and integrate their own job boards built on this open source platform. Future plans include updates for newer Drupal versions and expanding to other web frameworks like Django and WordPress.
Local Weather Information and GNOME Shell ExtensionSammy Fung
?
This document provides information about an upcoming presentation on local weather information and GNOME Shell extensions. The presentation will discuss obtaining weather data from local meteorological observatories and making it available as open data. It will also cover creating weather widgets for the GNOME Shell desktop environment. The presenter has 15+ years experience in open source communities and organizing conferences in Asia and the US.
This document provides information about Mozilla and the Mozilla community in Hong Kong. It lists Mozilla's products like Firefox and Thunderbird and ways to get involved through activities like coding, documentation, localization, and education. It also provides details on the Mozilla Hong Kong community group and past Firefox launch events in Hong Kong. Contact information is given for the Mozilla Hong Kong community liaison Sammy Fung.
Sammy Fung has over 15 years of experience in open source communities. He discusses open source marketing strategies in Hong Kong and Asia, including establishing communities, organizing events, and leveraging social media. Some examples of events he has organized include the first Linux User Group meetup in 1997 and the Hong Kong Open Source Conference in 2013 and 2014. He outlines marketing plans, elements like budgets and teams, and how to execute marketing through branding, media relations, and targeting audiences. The presentation focuses on his experience growing open source conferences like HKOSC, which saw a significant increase in overseas speakers and participants between 2013 and 2014.
DevRel - Transform article writing from printing to onlineSammy Fung
?
The document outlines the transformation of article writing from print to online focusing on developer relations (DevRel) and community engagement. It discusses collaborative interactions between companies and developers, community gatherings, conferences, and hackathons that foster business and technology activities. Additionally, it highlights the opportunities for new partnerships and ideas that emerge in the DevRel ecosystem.
Introduction to Open Source by opensource.hk (2019 Edition)Sammy Fung
?
OpenSource.hk is a community focused on promoting open source software in Hong Kong, supporting various contributions such as events and mentoring. It operates under a Creative Commons license and encourages collaboration and participation from volunteers. The document discusses open source software development processes, business models, and highlights various resources and educational materials related to open source.
My Open Source Journey - Developer and CommunitySammy Fung
?
Sammy Fung shares his journey in the open source community, highlighting his self-taught programming skills and contributions to various projects using languages like Python, Perl, and PHP. He emphasizes the value of open source software for personal and professional growth, detailing his roles in leading organizations like HKLUG and Open Source Hong Kong while navigating challenges in volunteer recruitment and community engagement. Fung advocates for balance between community service and personal income, urging others to support and contribute to the open source ecosystem.
Introduction to development with Django web frameworkSammy Fung
?
This document introduces Django web framework development. It discusses installing and setting up Django, creating a sample "eventorg" project with an "eventmgr" app, designing data models and accessing them, designing URLs and views, and designing templates. The goal is to teach the reader how to create a basic Django application from start to finish.
Sammy Fung discusses open data and web APIs in his presentation at Hong Kong Polytechnic University, emphasizing the importance of structured and machine-readable data. The presentation covers how to access data like air quality indexes using web scraping techniques and highlights the concept of open data deployments with a 5-star rating system. It also explains the role of APIs in data retrieval and touches upon the development and community around open source software.
Mozilla is a non-profit organization that promotes openness, innovation, and opportunity on the web through products like Firefox and Thunderbird. Its mission is to ensure the web remains open and accessible to all. Mozilla also develops technologies like Webmaker to teach people how to use and contribute to the open web. The presentation announces upcoming events in June 2015 on open source and Mozilla in Hong Kong to bring people together to learn and contribute to the open web.
This document summarizes Sammy Fung's involvement in open source technology and community over the past few decades. It notes that Sammy is a founding chairman of HKLUG, president of Open Source Hong Kong, and a member of GNOME Asia Committee. It then discusses Sammy's journey learning about and contributing to open source since 1991, including founding various open source organizations in Hong Kong. Finally, it encourages participating in open source communities to develop skills, networks, and have fun while contributing to society.
Access Open Data with Open Source Software ToolsSammy Fung
?
The document discusses using open data and open source software tools. It provides an overview of open data and open source, describing key concepts like freedoms for users. Popular open source stacks like LAMP are presented, along with tools like Python. The document outlines several of the author's past open data projects and describes a current project scraping Hong Kong restaurant license data using Scrapy, an open source web crawling framework. The project code is available on GitHub.
Installation of LAMP Server with Ubuntu 14.10 Server EditionSammy Fung
?
This document provides a comprehensive guide to installing a LAMP server using Ubuntu 14.10 server edition, detailing system requirements and alternatives. It covers configuration steps for Linux, Apache, MySQL, and PHP, alongside practical commands and tips for setting up a web server environment. Additionally, it includes insights on user management, networking, and database handling within the Linux operating system.
Software Freedom and Open Source CommunitySammy Fung
?
This document discusses open source software and the open source community. It describes key open source projects like Linux, Apache, Firefox, and their histories. It also outlines four essential freedoms of free software. Additionally, it discusses Hong Kong open source organizations and events that promote open source software, including OpenSource.HK and Hong Kong Creative Open Technology Association. Finally, it provides an overview of the Google Code-in and Google Summer of Code programs that introduce students to open source development.
This document discusses building a job site using the open source content management system Drupal. It describes how the speaker developed an open source Drupal module for job postings after encountering errors on an existing job site. The module allows users to post and categorize jobs and view category-specific RSS feeds. Instructions are provided on setting up a Linux web server and installing Drupal and relevant modules to build a customizable job board website.
Use open source software to develop ideas at workSammy Fung
?
The document discusses the importance of open source software in future market leadership, emphasizing the freedoms it provides to users and the community. It highlights the evolution of open source technologies, major contributions like Linux and Apache, and examples of open source projects such as a job board and weather data parser. The content also outlines opportunities for community involvement in open source through various organizations and events in Hong Kong.
Software Freedom Day 2014 in Hong Kong featured talks on free software and community, technology use, and contributions from various speakers. The event was organized by multiple groups including Open Source Hong Kong and the Hong Kong Linux User Group, with a focus on the four essential freedoms of free software. The document also highlights the development of key technologies such as the Linux operating system and Apache web server, alongside the importance of community engagement in open-source development.
Introduction of Mozilla Hong Kong (COSCUP 2014)Sammy Fung
?
Sammy Fung is a developer and Mozilla representative who serves as the Mozilla Hong Kong Community Liaison. He is the president of OpenSource.HK and director at HKCOTA. He participates in open source communities in Asia and works on projects like the zh_HK localization of Firefox and the Mozilla Hong Kong community website at mozilla.hk.
Introduction of Open Source Job Board with Drupal CMSSammy Fung
?
This document introduces an open source job board built with the Drupal content management system (CMS). It discusses how the job board allows for posting and categorizing jobs, company profiles, RSS feeds, and was developed using open source software. The job board code is available on GitHub and support services are offered to help organizations install, customize, and integrate their own job boards built on this open source platform. Future plans include updates for newer Drupal versions and expanding to other web frameworks like Django and WordPress.
Local Weather Information and GNOME Shell ExtensionSammy Fung
?
This document provides information about an upcoming presentation on local weather information and GNOME Shell extensions. The presentation will discuss obtaining weather data from local meteorological observatories and making it available as open data. It will also cover creating weather widgets for the GNOME Shell desktop environment. The presenter has 15+ years experience in open source communities and organizing conferences in Asia and the US.
This document provides information about Mozilla and the Mozilla community in Hong Kong. It lists Mozilla's products like Firefox and Thunderbird and ways to get involved through activities like coding, documentation, localization, and education. It also provides details on the Mozilla Hong Kong community group and past Firefox launch events in Hong Kong. Contact information is given for the Mozilla Hong Kong community liaison Sammy Fung.
Sammy Fung has over 15 years of experience in open source communities. He discusses open source marketing strategies in Hong Kong and Asia, including establishing communities, organizing events, and leveraging social media. Some examples of events he has organized include the first Linux User Group meetup in 1997 and the Hong Kong Open Source Conference in 2013 and 2014. He outlines marketing plans, elements like budgets and teams, and how to execute marketing through branding, media relations, and targeting audiences. The presentation focuses on his experience growing open source conferences like HKOSC, which saw a significant increase in overseas speakers and participants between 2013 and 2014.