PyCon 2015 Crawler Tutorial Explain Cookies

Jun 1, 20152 likes421 views

Wey-Han Liaw

20150530 Crawler Tutorial Introducing Cookies

Cookies
我們在參加PyCon的時候要先出?示通?行證場
務?人員會掃描QR code 然後給你識別證

Cookies
拿午餐便當的時候場務?人員會在識別證上劃記
表?示這個?人拿過便當了

Cookies
所以當他想再去拿?一個便當的時候 ?工作?人員就
知道他已經拿過便當了不能再發

HTTP Cookies
Username
/Password
Login
Set Cookie

HTTP Cookies
Username
/Password
Login
Set Cookie
Access to Page

Cookies
Cookie的作?用就像識別證?一樣
可以讓server知道你的身分

Cookies
Cookie通常不是給?人看的
Cookie裡的東西通常加密過
很難偽裝
很多網站還會在每次造訪網站時隨機?生key

Cookie Schema
?一個Cookie會有以下幾個屬性：
name
value
(domain)
(path)
(expires)

Cookie Schema
我們以識別證為例?子
name = “Pycon_ID”
value = “218”
domain = “中央研究院?人?文社會科學館”
path = “PYCON_APAC_2015”
expires = “Sat June 7 2015 18:00:00”

Cookies
Cookie 是 Client Only 的
Browser 會儲存這些cookie, 在發出request時
就會嵌在header裡
HTTP Response 可以在header裡傳入
Set-Cookie 告訴Client多加?一個Cookie

我们来解释?一下流程
PTT轉址
我同意此條款
已年滿??十八歲
POST over18

我们来解释?一下流程
PTT轉址
我同意此條款
已年滿??十八歲
POST over18
PTT set-cookie

我们来解释?一下流程
PTT轉址
我同意此條款
已年滿??十八歲
POST over18
PTT set-cookie
Access to Page

Hands-on Practice
?用Python requests抓取PTT八卦版資料！

Mantra is an application architecture specification and set of libraries for building Meteor apps. It is based on Flux principles and includes core components like React UI components, action handlers for business logic, state managers for local and remote data, and container components to connect UI, actions and states. While Mantra provides a good architecture, the author believes it could be improved by being tailored more specifically for Meteor, enhancing unit testing support, and developing a code generation tool or opinionated framework to implement the pattern.

Async, awaitWey-Han Liaw

This document discusses asynchronous programming in Python. It introduces the concepts of asynchronous I/O, event loops, and callbacks. Traditional approaches to asynchronous programming using callbacks are messy and hard to organize. Generators and coroutines provide a cleaner solution through yield and await keywords. The asyncio module implements asynchronous I/O with coroutines and provides an event loop to execute coroutine functions concurrently.

Udacity Taipei meetup #2 如何駕馭線上課程Wey-Han Liaw

NCCU 0617 talkWey-Han Liaw

SITCON summer camp 2015: Adrian說他那身邊的一堆社群Wey-Han Liaw

笔测迟丑辞苍自学从你小时候开始Wey-Han Liaw

Understanding Meteor StackWey-Han Liaw

The document discusses the key components that make up the Meteor stack, including Blaze, Tracker, DDP, and Mongo. It notes that while Meteor uses these components by default, the framework is customizable - developers can replace the UI library, database, and DDP client/server with alternatives. The standard Meteor application architecture is then outlined, explaining how each component fits together and interacts to provide reactivity on the client.

PyCon 2015 Crawler Tutorial Explain EncodingWey-Han Liaw

Implementation of Rubik's Cube Formula in PyCuberWey-Han Liaw

The document discusses PyCuber, a Python package for solving Rubik's Cubes. It introduces Rubik's Cube and its creator Erno Rubik. It then describes how PyCuber implements classes like Step and Formula to represent cube positions and solving algorithms. Various methods are included, like optimizing formulas and randomizing or mirroring solutions. Some interesting Rubik's Cube facts and records are also mentioned.

Intro to Apache Spark - LabMammoth Data

This document provides an introduction and overview of several Apache Spark labs covering: a "hello world" example of Resilient Distributed Datasets (RDDs); importing and performing operations on a wine dataset using DataFrames and SQL; and using the MLlib library to perform k-means clustering on features from the wine dataset. The labs demonstrate basic Spark concepts like RDDs, DataFrames, ML pipelines, and clustering algorithms.

Building custom kernels for IPythonNarahari (Hari) Allamraju

This document discusses building custom kernels for IPython. It begins by explaining what an IPython kernel is and how kernels work. It describes the benefits of building custom kernels for other languages that lack interactive development tools. The architecture of IPython is explained, with the kernel and clients communicating over ZeroMQ. The two types of kernels are discussed. The messaging specification and channels are covered at a high level. Finally, the document focuses on building wrapper kernels by extending the Kernel base class and implementing specific methods, using examples like a Bash and Redis kernel.

IPython & JupyterBig Data Colombia

Este documento describe la evolución de IPython y Jupyter, desde sus inicios como un shell interactivo de Python hasta convertirse en una plataforma multi-lenguaje para computación interactiva y publicación de documentos. Se explica cómo el protocolo REPL genérico de Jupyter permite ejecutar código en múltiples lenguajes y cómo herramientas como JupyterHub, nbviewer y notebooks han impulsado su adopción en educación, investigación y comunicación científica.

Spark Kernel Talk - Apache Spark Meetup San Francisco (July 2015)Robert "Chip" Senkbeil

NOTE: This was converted to Powerpoint from Keynote. 狠狠撸share does not play the embedded videos. You can download the powerpoint from slideshare and import it into keynote. The videos should work in the keynote. Abstract: In this presentation, we will describe the "Spark Kernel" which enables applications, such as end-user facing and interactive applications, to interface with Spark clusters. It provides a gateway to define and run Spark tasks and to collect results from a cluster without the friction associated with shipping jars and reading results from peripheral systems. Using the Spark Kernel as a proxy, applications can be hosted remotely from Spark.

Jupyter Kernel: How to Speak in Another LanguageWey-Han Liaw

The document discusses how to create a Jupyter kernel. It explains that kernels use ZeroMQ sockets to communicate with clients via the Jupyter messaging protocol. Native kernels are implemented from scratch while wrapper kernels are built from an existing interpreter using Python. The document provides examples of existing kernels like IJulia and the Python ipykernel. It also outlines the steps to build a wrapper kernel and mentions several other kernel types.