API Scraping. How to protect your API against something that is not necessarily an attack

Oct 27, 20200 likes332 views

API scraping involves extracting content from an API for use outside of its intended service. While it may seem like ordinary user behavior, it can have negative consequences for API owners through lost revenue, intellectual property issues, and reverse engineering risks. API owners should implement monitoring, throttling policies, and bot detection to protect their APIs from unwanted scraping activities without blocking legitimate uses. Completely preventing API scraping is difficult, but measures can be taken to make it harder.

API Scraping.
How to protect your API against something that is not
necessarily an attack
Artem Demchenkov
Illustration © Caterina Carraro/Billie

2
Artem Demchenkov
● CTO & Co-Founder at Billie
● ardemchenkov@gmail.com
● https://www.linkedin.com/in/artem-demchenkov-76b69934/

Agenda
3
● API scraping may not look like an attack
● Why should we worry then?
● KYCB
● How to recognize that you’re being scrapped
● How to stop API scraping. It is even possible?
● Bonus

Some numbers
5
● 46% of web traffic are scraping bots (Distil Networks, 2016)
● 2% of online revenue is lost due to web scraping (Distil Networks, 2016)
● Global e-commerce sales are 3.53 trillion dollars (Statista, 2019)
● 2% of it is >70 billion dollars

API Attacks
7
● DoS/DDoS
● Injections
● Data Exposure
● Authentication Hijacking
● “Man in the Middle”
● Unencrypted Communication
● Application Abuse
● Parameter Tampering
}Unexpected API usage

API Scraping
9
● Is a pretty ordinary and expected user behaviour
● It doesn’t try to hijack or break anything
● Content - is the main point of interest
● That’s why it is not always easy to identify that you are being scraped

Ordinary user behavior ﬂow
10
USER UI API DATA SOURCE

Ordinary user behavior ﬂow
11
USER UI API DATA SOURCE

Google Maps Terms of Service
14
3.2.3 Restrictions Against Misusing the Services.
(a) No Scraping. Customer will not export, extract, or otherwise scrape Google Maps
Content for use outside the Services. For example, Customer will not: (i) pre-fetch, index,
store, reshare, or rehost Google Maps Content outside the services; (ii) bulk download
Google Maps tiles, Street View images, geocodes, directions, distance matrix results, roads
information, places information, elevation values, and time zone details; (iii) copy and save
business names, addresses, or user reviews; or (iv) use Google Maps Content with
text-to-speech services.

15
But not everyone is Google, isn’t it?

Scraping ﬂow
16
USER UI API DATA SOURCEBOT

Scraping communication ﬂow
17
USER UI API EXTERNAL APIBOT

What makes API Scraping dangerous for API Owners
18
● Unpredictable expenses
● Income losses
● Loss of intellectual property ownership
● Loss of competitive advantage
● Risk of being reverse engineered

How to recognize that your API is being scrapped?
20
Logging Thresholds Monitoring alerts

How to set up a proper Data Monitoring
Smart Data-driven Alerts with Prometheus and Grafana
https://youtu.be/GbwyF6xZwwc

How to prevent API Scraping
22
HTTP-Request limits Per user, Per time period
Basic Firewall rules Headers, Content-Size, IPs
More extended rules Geography, Pattern Detection
IP databases https://www.abuseipdb.com/
Real-time ML based bot detection Specific services (e.g. Cloudflare)
Data thresholds Num or registrations, num of API calls...

23
They say:
no one can block API scraping completely

24
We say:
it is a race and one who stops ﬁrst loses

Scraping communication ﬂow
26
USER UI API DATA SOURCEBOTHUMAN
This one leaves traces

27
Prior to building a bot,
a human needs to explore your API

Thank you
● ardemchenkov@gmail.com
● https://www.linkedin.com/in/artem-demchenkov-76b69934/

Check it out. There’s interesting
More in our “Engineering Corner” on Medium
29
https://medium.com/billie-ﬁnanzratgeber

API scraping involves automated extraction of content from APIs for purposes outside of their intended use. While not necessarily malicious, it can harm API owners through lost revenue and intellectual property. To recognize scraping, API owners monitor for abnormal usage patterns and set limits on requests per user. Methods to prevent scraping include rate limits, firewall rules blocking suspicious traffic, IP databases, and machine learning models to detect bots. However, completely blocking scraping may not be possible due to the dynamic nature of the problem.

Microservics, serverless and real time; Building blocks of the modern data pi...Manisha Sule

��

The document discusses modern approaches for building data pipelines, including microservices, serverless computing, and real-time analytics. It describes how microservices break large applications into independent services, and the advantages this provides. Serverless computing is defined as an event-driven execution model where cloud providers manage infrastructure. Real-time analytics involves processing and analyzing data as it becomes available. Examples of AWS and Google Cloud services for implementing real-time analytics are also provided.

apidays Paris 2024 - Do not Live in the Shadow (APIs) - Teresa Pereira, Sieme...apidays

��

Do not live in the Shadow (APIs) Teresa Pereira, Threat Hunter at Siemens Energy apidays Paris 2024 - The Future API Stack for Mass Innovation December 3 - 5, 2024 ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

5 Pillars of Building Enterprise0grade APIsWSO2

��

Any new digital service being built today also needs to be exposed as an API. This is the core of agile, successful digital businesses. It forces digital organizations to create new APIs while consuming many other APIs in the process, effectively being part of the global API supply chain. However, many API strategies fail, mostly due to underestimating the full lifecycle of APIs from conceptualization to engineering to production and evolution. Getting optimal ROI from APIs requires understanding the nuances of building APIs and finding the right balance between what you build and reuse. This slide deck discusses: • How we develop APIs today and commonly noticed problems • The different types of APIs in an organization and their nuances • 5 key elements for developing enterprise-grade APIs for the enterprise • The safest bet for a successful API strategy We also explore Choreo, an integration Platform as a Service for API developers: https://wso2.com/choreo

[WSO2 Integration Summit San Francisco 2019] Protecting API Infrastructures —...WSO2

��

What do Google, Facebook, Paypal, IRS, T-mobile, and USPS have in common? Answer -- hackers used their APIs to access sensitive customer information. Although these API attacks were exposed, most API-based attacks go undetected these days. This deck will discuss today’s evolving API threat landscape and explore what you can do to both detect and block cyberattacks from authenticated users and hackers who have reverse engineered your API with an integrated solution from WSO2 and PingIntelligence.

[APIdays INTERFACE 2021] Programming the Cloud through APIsWSO2

��

We live in a world where many digital services are offered as APIs today. You will find APIs for finance, healthcare, communication, retail, travel and transportation, leisure, real estate, and many more services. These APIs form a large global ecosystem of APIs providing great value for new digital services. For businesses to be able to participate in this ecosystem, they have to be providing their services through the Cloud, in the form of APIs. This workshop discusses what developers should be thinking about when developing and exposing new APIs on the cloud. It will also cover best practices, pitfalls, and security-related matters that will be important for developers and architects who are building APIs on the cloud.

Jim Novack in #awe2015 with Talent Swarm. Best practices tipsDYNATEC S.A. Especialistas en Proyectos de Ingenieria a nivel Mundial

��

Airbnb is rolling out a new identity verification program that will require all users to connect their online identity with a government ID. Users can confirm their birthdate and partial social security number or upload a scanned ID linked to their Airbnb, Facebook, or LinkedIn profile. This will display a "Verified ID" badge. Airbnb says trust is key to their community and anonymity is not allowed. They will start requiring ID verification for some US users today and expand globally over time.

IRJET- Phishing Website Detection based on Machine LearningIRJET Journal

��

This document proposes a machine learning model to detect phishing websites. It discusses how data mining algorithms can be used to classify websites as legitimate or phishing based on their characteristics. The proposed system aims to optimize detection by analyzing URL features, checking blacklists, and using a WHOIS database. It claims this method could decrease the error rate of existing detection systems by 30% and provide a more efficient way to identify phishing websites.

API Security Best Practices and GuidelinesWSO2

��

View on-demand: https://wso2.com/library/webinars/api-security-best-practices-and-guidelines/ Modern enterprises are increasingly adopting APIs, exceeding all predictions. With more businesses investing in microservices and the increased consumption of cloud APIs, you need to secure beyond just a handful of well-known APIs. You will need to secure a higher number of internal and external endpoints. At the same time, security itself is a broad area and vendors implement a number of seemingly similar standards and patterns, making it very difficult for consumers to settle on the best option for securing APIs. The sheer number of options can be very confusing. There is much to learn about API security, regardless of whether you are a novice or expert and it’s extremely important that you do because security is an integral part of any development project, including API ecosystems. This webinar will deep-dive into the importance of API security, API security patterns, and how identity and access management (IAM) fit in the ecosystem. DURING THE WEBINAR, WE WILL COVER: Managed APIs OAuth 2.0 and API security patterns Introduction to WSO2 Identity Server How we align with OWASP API security guidelines

apidays Helsinki & North 2023 - API Security in the era of Generative AI, Mat...apidays

��

apidays Helsinki & North 2023 API Ecosystems - Connecting Physical and Digital June 5 & 6, 2023 API Security in the era of Generative AI Matt Feigal, Partner Engineering Manager at Google Cloud Sweden ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

FireTail at API Days Australia 2024 - The Double-edge sword of AI for API Sec...JeremySnyder8

��

architecting-ai-in-the-enterprise-apis-and-applications.pdfWSO2

��

Top OSS for Mobile AppSec Testing: The Latest on R2 and FRIDANowSecure

��

IRJET- 3 - Tier Cross Platform Application for Digital Visiting Card with...IRJET Journal

��

This document proposes developing a 3-tier cross-platform application for digital visiting cards with a graphical user interface. The application would allow users to create, edit, delete, share and connect digital business cards through a web or mobile application. A cloud database would store the data and create RESTful APIs to allow access from different platforms. The digital cards could be graphically designed and shared across various social media and messaging platforms. The goal is to eliminate physical business cards and make the process of creating and sharing cards more user-friendly and accessible across different devices.

Design - Start Your API Journey TodayLaurenWendler

��

This document discusses how APIs can power the API economy and digital transformation. It describes how APIs can be used to expose functionality from various systems and data sources, and how API management is needed to securely manage APIs. The document promotes IBM API Connect as a way to create, manage and secure APIs and microservices across hybrid cloud environments to drive innovation. It provides links to learn more about API Connect and digital transformation workshops.

2016: The Year to Align Marketing & IT DepartmentsYottaa

��

Understanding the importance of marketing and IT alignment is one thing, but setting a plan for execution is a difficult challenge for any enterprise to undertake. This process involves disrupting well-established processes and embracing new, often radical, ideas around how separate teams can work together to deliver web experiences that match the sky-high expectations of the modern Internet user. These slides explore: Prescriptive advice to balance engagement & performance Identification of problematic vendor technologies before they wreak havoc on your site Proper strategy development to apply personalization based on user context How IT and Marketers must work together to achieve faster and more profitable websites

The Best of Both Worlds: Introducing WSO2 API Manager 4.0.0 [ANZ]WSO2

��

DURING THE WEBINAR, WE WILL COVER: Experience the power and synergy of Service Integration and API Management in a fully functional API ecosystem Understand the motivation behind WSO2 API Manager 4.0.0 release New streaming and event-driven architecture support available in API Manager 4.0.0 Learn the importance of catering all API Management and integration demands with one connected platform Explore other new features and enhancements to the product

[WSO2 Integration Summit Singapore 2019] Transforming Your Business through APIsWSO2

��

Cyber security series Application SecurityJim Kaplan CIA CFE

��

This webinar series is designed to help internal auditors looking to equip themselves with competencies and confidence to handle audit of IT controls and information security, and learn about the emerging technologies and their underlying risks The series focuses on contemporary IT audit approaches relevant to Internal Auditors and the processes underlying risk based IT audits. Session 6 of 10 This Webinar focuses on Application Security • Application security logging and monitoring • Issues in current logging practices • Resources required by developers for security logging • Correlating and alerting from log sources • Logging in multi-tiered architectures and disparate systems • Application security logging requirements

Find IT & Marketing’s Common Ground: Make Your Site FasterGhostery, Inc.

��

This document discusses how marketing and IT departments often have misaligned incentives and goals, which can negatively impact business metrics like conversion rates and customer satisfaction. It presents data showing that website speed and privacy/security issues directly impact key performance indicators. The document proposes that the marketing cloud company Ghostery and digital experience optimization company Yottaa can help bridge the gaps between marketing and IT by providing tools to improve website performance, streamline third-party tags and scripts, and ensure privacy and compliance. Case studies demonstrate how these companies' solutions have boosted metrics for customers like A Place for Mom and Moosejaw.

SF Big Analytics Meetup - Exact Count Distinct with Apache KylinSamanthaBerlant

��

With over 450 million customers, Didi (world’s largest rideshare company) conducts complex user behavior analysis on huge datasets daily. Exact Count Distinct is one of Didi’s most critical metrics, but it is known for being computationally heavy and notoriously slow. The difference between exact Count Distinct and approximate Count Distinct can cost Didi millions of dollars. In this talk, Kaige Liu of the Apache Kylin project will explain how Didi uses Apache Kylin to return exact Distinct Count on billions of rows of data with sub-second latency to generate the most accurate picture of its business. You will also learn about the latest development in modern OLAP technologies. Kaige will share how Didi and Truck Alliance (a truck-hailing company that processes $100 billion worth of goods yearly) use Apache Kylin to power their analytics platforms that allow 100s of analysts to achieve sub-second latency on petabyte-scale data.

API Gateway How-To: The Many Ways to Apply the Gateway PatternVMware Tanzu

��

The document discusses different patterns for using an API gateway, including: 1) API management for existing APIs to handle cross-cutting concerns in one place 2) Ingress for APIs and services to manage traffic entering a cluster 3) Application modernization to help split a monolith into microservices gradually 4) Simplified aggregated API to simplify access to multiple complex internal services 5) Using a service mesh for internal east-west traffic between services while using a gateway for north-south external traffic The document provides examples and use cases for each pattern and concludes that the right approach depends on specific needs, and that gateways and service meshes can overlap on common concerns like security and observability.

INTERFACE by apidays 2023 - Everything you need to know about API security, T...apidays

��

INTERFACE by apidays 2023 APIs for a “Smart” economy. Embedding AI to deliver Smart APIs and turn into an exponential organization June 28 & 29, 2023 Everything you need to know about API security Tony Lauro, Director of Security Strategy at Akamai Technologies ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

How to Guarantee Exact COUNT DISTINCT Queries with Sub-Second Latency on Mass...Tyler Wishnoff

��

How to Guarantee Exact Count Distinct Queries with Sub-Second Latency on Mass...SamanthaBerlant

��

Design - Start Your API Journey TodayLaurenWendler

��

This document discusses the opportunities presented by the API economy. It describes how APIs can enable new experiences like pre-ordering food from a restaurant app before arriving. The document outlines challenges of managing APIs at scale across hybrid cloud environments. It introduces IBM API Connect as a way to create, manage and secure APIs throughout their lifecycle to power digital transformations. Finally, it provides links to learn more about API Connect and taking advantage of IBM's cloud offerings.

Zymr Fintech app developmentZymr Inc

��

This document discusses best practices for developing FinTech apps. It covers fundamental practices like ensuring information security and compliance with PCI DSS standards, eliminating vulnerabilities, integrating fraud detection using machine learning, maintaining proper testing coverage, and using microservices architecture. It also discusses current FinTech trends like the increasing use of blockchain, artificial intelligence, and quantum computing. Finally, it provides examples of projects completed by Zymr, including developing a mobile commerce solution and a digital payments risk management platform.

CASB — Your new best friend for safe cloud adoption? Digital Transformation EXPO Event Series

��

Unit II: Design of Static Equipment FoundationsSanjivani College of Engineering, Kopargaon

��

How to Make an RFID Door Lock System using ArduinoCircuitDigest

��

More Related Content

Similar to API Scraping. How to protect your API against something that is not necessarily an attack (20)

API Security Best Practices and GuidelinesWSO2

��

apidays Helsinki & North 2023 - API Security in the era of Generative AI, Mat...apidays

��

FireTail at API Days Australia 2024 - The Double-edge sword of AI for API Sec...JeremySnyder8

��

architecting-ai-in-the-enterprise-apis-and-applications.pdfWSO2

��

Top OSS for Mobile AppSec Testing: The Latest on R2 and FRIDANowSecure

��

IRJET- 3 - Tier Cross Platform Application for Digital Visiting Card with...IRJET Journal

��

Design - Start Your API Journey TodayLaurenWendler

��

2016: The Year to Align Marketing & IT DepartmentsYottaa

��

The Best of Both Worlds: Introducing WSO2 API Manager 4.0.0 [ANZ]WSO2

��

[WSO2 Integration Summit Singapore 2019] Transforming Your Business through APIsWSO2

��

Cyber security series Application SecurityJim Kaplan CIA CFE

��

Find IT & Marketing’s Common Ground: Make Your Site FasterGhostery, Inc.

��

SF Big Analytics Meetup - Exact Count Distinct with Apache KylinSamanthaBerlant

��

API Gateway How-To: The Many Ways to Apply the Gateway PatternVMware Tanzu

��

INTERFACE by apidays 2023 - Everything you need to know about API security, T...apidays

��

How to Guarantee Exact COUNT DISTINCT Queries with Sub-Second Latency on Mass...Tyler Wishnoff

��

How to Guarantee Exact Count Distinct Queries with Sub-Second Latency on Mass...SamanthaBerlant

��

Design - Start Your API Journey TodayLaurenWendler

��

Zymr Fintech app developmentZymr Inc

��

CASB — Your new best friend for safe cloud adoption? Digital Transformation EXPO Event Series

��

API Security Best Practices and GuidelinesWSO2

��

apidays Helsinki & North 2023 - API Security in the era of Generative AI, Mat...apidays

��

FireTail at API Days Australia 2024 - The Double-edge sword of AI for API Sec...JeremySnyder8

��

architecting-ai-in-the-enterprise-apis-and-applications.pdfWSO2

��

Top OSS for Mobile AppSec Testing: The Latest on R2 and FRIDANowSecure

��

IRJET- 3 - Tier Cross Platform Application for Digital Visiting Card with...IRJET Journal

��

Design - Start Your API Journey TodayLaurenWendler

��

2016: The Year to Align Marketing & IT DepartmentsYottaa

��

The Best of Both Worlds: Introducing WSO2 API Manager 4.0.0 [ANZ]WSO2

��

[WSO2 Integration Summit Singapore 2019] Transforming Your Business through APIsWSO2

��

Cyber security series Application SecurityJim Kaplan CIA CFE

��

Find IT & Marketing’s Common Ground: Make Your Site FasterGhostery, Inc.

��

SF Big Analytics Meetup - Exact Count Distinct with Apache KylinSamanthaBerlant

��

API Gateway How-To: The Many Ways to Apply the Gateway PatternVMware Tanzu

��

INTERFACE by apidays 2023 - Everything you need to know about API security, T...apidays

��

How to Guarantee Exact COUNT DISTINCT Queries with Sub-Second Latency on Mass...Tyler Wishnoff

��

How to Guarantee Exact Count Distinct Queries with Sub-Second Latency on Mass...SamanthaBerlant

��

Design - Start Your API Journey TodayLaurenWendler

��

Zymr Fintech app developmentZymr Inc

��

CASB — Your new best friend for safe cloud adoption? Digital Transformation EXPO Event Series

��

Recently uploaded (20)

Unit II: Design of Static Equipment FoundationsSanjivani College of Engineering, Kopargaon

��

How to Make an RFID Door Lock System using ArduinoCircuitDigest

��

Frankfurt University of Applied Science urkundeLisa Emerson

��

AI, Tariffs and Supply Chains in Knowledge GraphsMax De Marzi

��

Taykon-Kalite belgeleriTAYKON

��

US Patented ReGenX Generator, ReGen-X Quatum Motor EV Regenerative Accelerati...Thane Heins NOBEL PRIZE WINNING ENERGY RESEARCHER

��

Preface: The ReGenX Generator innovation operates with a US Patented Frequency Dependent Load Current Delay which delays the creation and storage of created Electromagnetic Field Energy around the exterior of the generator coil. The result is the created and Time Delayed Electromagnetic Field Energy performs any magnitude of Positive Electro-Mechanical Work at infinite efficiency on the generator's Rotating Magnetic Field, increasing its Kinetic Energy and increasing the Kinetic Energy of an EV or ICE Vehicle to any magnitude without requiring any Externally Supplied Input Energy. In Electricity Generation applications the ReGenX Generator innovation now allows all electricity to be generated at infinite efficiency requiring zero Input Energy, zero Input Energy Cost, while producing zero Greenhouse Gas Emissions, zero Air Pollution and zero Nuclear Waste during the Electricity Generation Phase. In Electric Motor operation the ReGen-X Quantum Motor now allows any magnitude of Work to be performed with zero Electric Input Energy. Demonstration Protocol: The demonstration protocol involves three prototypes; 1. Protytpe #1, demonstrates the ReGenX Generator's Load Current Time Delay when compared to the instantaneous Load Current Sine Wave for a Conventional Generator Coil. 2. In the Conventional Faraday Generator operation the created Electromagnetic Field Energy performs Negative Work at infinite efficiency and it reduces the Kinetic Energy of the system. 3. The Magnitude of the Negative Work / System Kinetic Energy Reduction (in Joules) is equal to the Magnitude of the created Electromagnetic Field Energy (also in Joules). 4. When the Conventional Faraday Generator is placed On-Load, Negative Work is performed and the speed of the system decreases according to Lenz's Law of Induction. 5. In order to maintain the System Speed and the Electric Power magnitude to the Loads, additional Input Power must be supplied to the Prime Mover and additional Mechanical Input Power must be supplied to the Generator's Drive Shaft. 6. For example, if 100 Watts of Electric Power is delivered to the Load by the Faraday Generator, an additional >100 Watts of Mechanical Input Power must be supplied to the Generator's Drive Shaft by the Prime Mover. 7. If 1 MW of Electric Power is delivered to the Load by the Faraday Generator, an additional >1 MW Watts of Mechanical Input Power must be supplied to the Generator's Drive Shaft by the Prime Mover. 8. Generally speaking the ratio is 2 Watts of Mechanical Input Power to every 1 Watt of Electric Output Power generated. 9. The increase in Drive Shaft Mechanical Input Power is provided by the Prime Mover and the Input Energy Source which powers the Prime Mover. 10. In the Heins ReGenX Generator operation the created and Time Delayed Electromagnetic Field Energy performs Positive Work at infinite efficiency and it increases the Kinetic Energy of the system.

Mathematics behind machine learning INT255 INT255__Unit 3__PPT-1.pptxppkmurthy2006

��

Turbocor Product and Technology Review.pdfTotok Sulistiyanto

��

Integration of Additive Manufacturing (AM) with IoT : A Smart Manufacturing A...ASHISHDESAI85

��

Combining 3D printing with Internet of Things (IoT) enables the creation of smart, connected, and customizable objects that can monitor, control, and optimize their performance, potentially revolutionizing various industries. oT-enabled 3D printers can use sensors to monitor the quality of prints during the printing process. If any defects or deviations from the desired specifications are detected, the printer can adjust its parameters in real time to ensure that the final product meets the required standards.

04 MAINTENANCE OF CONCRETE PAVEMENTS.pptsreenath seenu

��

Industrial Valves, Instruments Products Profilezebcoeng

��

We’re excited to share our product profile, showcasing our expertise in Industrial Valves, Instrumentation, and Hydraulic & Pneumatic Solutions. We also supply API-approved valves from globally trusted brands, ensuring top-notch quality and internationally certified solutions. Let’s explore valuable business opportunities together! We specialize in: • Industrial Valves (Gate, Globe, Ball, Butterfly, Check) • Instrumentation (Pressure Gauges, Transmitters, Flow Meters) • Pneumatic Products (Cylinders, Solenoid Valves, Fittings) As authorized partners of trusted global brands, we deliver high-quality solutions tailored to meet your industrial needs with seamless support.

US Patented ReGenX Generator, ReGen-X Quatum Motor EV Regenerative Accelerati...Thane Heins NOBEL PRIZE WINNING ENERGY RESEARCHER

��

How Engineering Model Making Brings Designs to Life.pdfMaadhu Creatives-Model Making Company

��

Optimization of Cumulative Energy, Exergy Consumption and Environmental Life ...J. Agricultural Machinery

��

Optimal use of resources, including energy, is one of the most important principles in modern and sustainable agricultural systems. Exergy analysis and life cycle assessment were used to study the efficient use of inputs, energy consumption reduction, and various environmental effects in the corn production system in Lorestan province, Iran. The required data were collected from farmers in Lorestan province using random sampling. The Cobb-Douglas equation and data envelopment analysis were utilized for modeling and optimizing cumulative energy and exergy consumption (CEnC and CExC) and devising strategies to mitigate the environmental impacts of corn production. The Cobb-Douglas equation results revealed that electricity, diesel fuel, and N-fertilizer were the major contributors to CExC in the corn production system. According to the Data Envelopment Analysis (DEA) results, the average efficiency of all farms in terms of CExC was 94.7% in the CCR model and 97.8% in the BCC model. Furthermore, the results indicated that there was excessive consumption of inputs, particularly potassium and phosphate fertilizers. By adopting more suitable methods based on DEA of efficient farmers, it was possible to save 6.47, 10.42, 7.40, 13.32, 31.29, 3.25, and 6.78% in the exergy consumption of diesel fuel, electricity, machinery, chemical fertilizers, biocides, seeds, and irrigation, respectively.

Engineering at Lovely Professional University (LPU).pdfSona

��

How to Build a Maze Solving Robot Using ArduinoCircuitDigest

��

Lessons learned when managing MySQL in the CloudIgor Donchovski

��

Managing MySQL in the cloud introduces a new set of challenges compared to traditional on-premises setups, from ensuring optimal performance to handling unexpected outages. In this article, we delve into covering topics such as performance tuning, cost-effective scalability, and maintaining high availability. We also explore the importance of monitoring, automation, and best practices for disaster recovery to minimize downtime.

CS3451-OPERATING-SYSTEM NOTES ALL123.pdfPonniS7

��

Cyber Security_ Protecting the Digital World.pptxHarshith A S

��

Env and Water Supply Engg._Dr. Hasan.pdfMahmudHasan747870

��

Unit II: Design of Static Equipment FoundationsSanjivani College of Engineering, Kopargaon

��

How to Make an RFID Door Lock System using ArduinoCircuitDigest

��

Frankfurt University of Applied Science urkundeLisa Emerson

��

AI, Tariffs and Supply Chains in Knowledge GraphsMax De Marzi

��

Taykon-Kalite belgeleriTAYKON

��

US Patented ReGenX Generator, ReGen-X Quatum Motor EV Regenerative Accelerati...Thane Heins NOBEL PRIZE WINNING ENERGY RESEARCHER

��

Mathematics behind machine learning INT255 INT255__Unit 3__PPT-1.pptxppkmurthy2006

��

Turbocor Product and Technology Review.pdfTotok Sulistiyanto

��

Integration of Additive Manufacturing (AM) with IoT : A Smart Manufacturing A...ASHISHDESAI85

��

04 MAINTENANCE OF CONCRETE PAVEMENTS.pptsreenath seenu

��

Industrial Valves, Instruments Products Profilezebcoeng

��

US Patented ReGenX Generator, ReGen-X Quatum Motor EV Regenerative Accelerati...Thane Heins NOBEL PRIZE WINNING ENERGY RESEARCHER

��

How Engineering Model Making Brings Designs to Life.pdfMaadhu Creatives-Model Making Company

��

Optimization of Cumulative Energy, Exergy Consumption and Environmental Life ...J. Agricultural Machinery

��

Engineering at Lovely Professional University (LPU).pdfSona

��

How to Build a Maze Solving Robot Using ArduinoCircuitDigest

��

Lessons learned when managing MySQL in the CloudIgor Donchovski

��

CS3451-OPERATING-SYSTEM NOTES ALL123.pdfPonniS7

��

Cyber Security_ Protecting the Digital World.pptxHarshith A S

��

Env and Water Supply Engg._Dr. Hasan.pdfMahmudHasan747870

��

API Scraping. How to protect your API against something that is not necessarily an attack

1. API Scraping. How to protect your API against something that is not necessarily an attack Artem Demchenkov Illustration © Caterina Carraro/Billie

2. 2 Artem Demchenkov ● CTO & Co-Founder at Billie ● ardemchenkov@gmail.com ● https://www.linkedin.com/in/artem-demchenkov-76b69934/

3. Agenda 3 ● API scraping may not look like an attack ● Why should we worry then? ● KYCB ● How to recognize that you’re being scrapped ● How to stop API scraping. It is even possible? ● Bonus

4. 4 Let’s begin with numbers

5. Some numbers 5 ● 46% of web traffic are scraping bots (Distil Networks, 2016) ● 2% of online revenue is lost due to web scraping (Distil Networks, 2016) ● Global e-commerce sales are 3.53 trillion dollars (Statista, 2019) ● 2% of it is >70 billion dollars

6. 6 API Attacks

7. API Attacks 7 ● DoS/DDoS ● Injections ● Data Exposure ● Authentication Hijacking ● “Man in the Middle” ● Unencrypted Communication ● Application Abuse ● Parameter Tampering }Unexpected API usage

8. 8 API Scraping is different

9. API Scraping 9 ● Is a pretty ordinary and expected user behaviour ● It doesn’t try to hijack or break anything ● Content - is the main point of interest ● That’s why it is not always easy to identify that you are being scraped

10. Ordinary user behavior ﬂow 10 USER UI API DATA SOURCE

11. Ordinary user behavior ﬂow 11 USER UI API DATA SOURCE

12. Scraping ﬂow 12 USER UI API DATA SOURCE

13. 13 Scraping is not allowed

14. Google Maps Terms of Service 14 3.2.3 Restrictions Against Misusing the Services. (a) No Scraping. Customer will not export, extract, or otherwise scrape Google Maps Content for use outside the Services. For example, Customer will not: (i) pre-fetch, index, store, reshare, or rehost Google Maps Content outside the services; (ii) bulk download Google Maps tiles, Street View images, geocodes, directions, distance matrix results, roads information, places information, elevation values, and time zone details; (iii) copy and save business names, addresses, or user reviews; or (iv) use Google Maps Content with text-to-speech services.

15. 15 But not everyone is Google, isn’t it?

16. Scraping ﬂow 16 USER UI API DATA SOURCEBOT

17. Scraping communication ﬂow 17 USER UI API EXTERNAL APIBOT

18. What makes API Scraping dangerous for API Owners 18 ● Unpredictable expenses ● Income losses ● Loss of intellectual property ownership ● Loss of competitive advantage ● Risk of being reverse engineered

19. 19 KYCB Know Your Customers’ Behavior

20. How to recognize that your API is being scrapped? 20 Logging Thresholds Monitoring alerts

21. How to set up a proper Data Monitoring Smart Data-driven Alerts with Prometheus and Grafana https://youtu.be/GbwyF6xZwwc

22. How to prevent API Scraping 22 HTTP-Request limits Per user, Per time period Basic Firewall rules Headers, Content-Size, IPs More extended rules Geography, Pattern Detection IP databases https://www.abuseipdb.com/ Real-time ML based bot detection Specific services (e.g. Cloudflare) Data thresholds Num or registrations, num of API calls...

23. 23 They say: no one can block API scraping completely

24. 24 We say: it is a race and one who stops ﬁrst loses

25. 25 Bonus Track

26. Scraping communication ﬂow 26 USER UI API DATA SOURCEBOTHUMAN This one leaves traces

27. 27 Prior to building a bot, a human needs to explore your API

28. Thank you ● ardemchenkov@gmail.com ● https://www.linkedin.com/in/artem-demchenkov-76b69934/

29. Check it out. There’s interesting More in our “Engineering Corner” on Medium 29 https://medium.com/billie-ﬁnanzratgeber

�ݺ�ߣ

API Scraping. How to protect your API against something that is not necessarily an attack

Recommended

More Related Content

Similar to API Scraping. How to protect your API against something that is not necessarily an attack (20)

Recently uploaded (20)

API Scraping. How to protect your API against something that is not necessarily an attack