Python for AI: A Crash Course

Nov 16 2024 · Python 3.12, JupyterLab 4.2.4

Lesson 05: Working with Online Data & APIs

Getting Online Data Demo

Episode complete

Play next episode

Next

Heads up... You’re accessing parts of this content for free, with some sections shown as obfuscated text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Heads up... You’re accessing parts of this content for free, with some sections shown as obfuscated text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Demo

Retrieving Information from Web Resources

You may occasionally need to GET the contents of web pages to “scrape” text information from sites that wouldn’t otherwise be accessible via an API. Fortunately, the requests library is equally useful for retrieving the contents of web pages since it’s for accessing APIs.

GETting a Web Page from a URL

Start with the kind of GET request that billions of people use daily: a request for the contents of a web page. In this case, we’ll use a GET request to retrieve the contents of Kodeco’s home page.

import requests

response = requests.get("https://kodeco.com/")

if response.status_code == 200:
  print("Successfully retrieved the web page!")
  print(f"Status code: {response.status_code}.")
  print(f"Content type: {response.headers["Content-Type"]}.")
  print("=====")
  print(response.text)
else:
  print(f"Failed to retrieve the web page. Status code: {response.status_code}.")

Successfully retrieved the web page!
Status code: 200.
Content type: text/html; charset=utf-8.
=====
<!DOCTYPE html>

(A whole lot of HTML goes here.)

GETting a Web Page from an Invalid URL

What happens when you try to GET from a URL that doesn’t correspond to a web resource? You can find out by changing the URL in the previous code to one for a resource that doesn’t exist.

import requests

response = requests.get("https://kodeco.com/non-existent-page")

if response.status_code == 200:
  print("Successfully retrieved the web page!")
  print(f"Status code: {response.status_code}.")
  print(f"Content type: {response.headers["Content-Type"]}.")
  print("=====")
  print(response.text)
else:
  print(f"Failed to retrieve the web page. Status code: {response.status_code}.")

Failed to retrieve the web page. Status code: 404.

Making the Simplest Possible GET Request

The Star Wars API is fun to play with. It’s an API that acts as an encyclopedia of Star Wars people, species, planets, vehicles, spaceships, and films, or at least episodes one through seven. You don’t need to pay or register for an API key to use it — you can start making calls to it. The API has a website at swapi.dev and its base URL is swapi.dev/api.

import json

SWAPI_BASE_URL = "https://swapi.dev/api/"
response = requests.get(SWAPI_BASE_URL)

if response.status_code == 200:
  data = response.json()
  pretty_json = json.dumps(data, indent=4, sort_keys=True)
  print(pretty_json)

Making a GET Request with Path Parameters

Use one of the Star Wars API endpoints: “people”.

PEOPLE_ENDPOINT = "people/"
person_id = 1
response = requests.get(f"{SWAPI_BASE_URL}{PEOPLE_ENDPOINT}{person_id}")

if response.status_code == 200:
  data = response.json()
  pretty_json = json.dumps(data, indent=4)
  print(pretty_json)

PEOPLE_ENDPOINT = "people/"
person_id = 1
person_response = requests.get(f"{SWAPI_BASE_URL}{PEOPLE_ENDPOINT}{person_id}")

if person_response.status_code == 200:
  person_data = person_response.json()
  homeworld_endpoint = person_data["homeworld"]
  homeworld_response = requests.get(homeworld_endpoint)
  if homeworld_response.status_code == 200:
    homeworld_data = homeworld_response.json()
    pretty_json = json.dumps(homeworld_data, indent=4)
    print(pretty_json)

Making a GET Request with Query Parameters

Now, take a look at an API that uses query parameters, where you provide parameters in the form of a query string at the end of the endpoint URL. The API in question is the Open-Meteo weather API (meteo is the French word for “weather”). Like the Star Wars API, the Open-Meteo API is free of charge and doesn’t require registering for an API key.

WEATHER_ENDPOINT = "https://api.open-meteo.com/v1/forecast"
parameters = {
  "latitude":  51.5072,
  "longitude": -0.1276,
  "current":   "weathercode,temperature_2m,relativehumidity_2m",
}
response = requests.get(WEATHER_ENDPOINT, params=parameters)

if response.status_code == 200:
  data = response.json()
  pretty_json = json.dumps(data, indent=4)
  print(pretty_json)

WEATHER_CODE_TABLE = {
  0: "clear sky",
  1: "mainly clear",
  2: "partly cloudy",
  3: "overcast",
  45: "fog",
  48: "depositing rime fog",
  51: "light drizzle",
  53: "moderate drizzle",
  55: "dense drizzle",
  56: "light freezing drizzle",
  57: "dense freezing drizzle",
  61: "slight rain",
  63: "moderate rain",
  65: "heavy rain",
  66: "light freezing rain",
  67: "heavy freezing rain",
  71: "slight snow",
  73: "moderate snow",
  75: "heavy snow",
  77: "snow grains",
  80: "light rain showers",
  81: "moderate rain showers",
  82: "violent rain showers",
  85: "slight snow showers",
  86: "heavy snow showers",
  95: "thunderstorm",
  96: "thunderstorm with slight hail",
  99: "thunderstorm with heavy hail",
}

print(f"The weather in London is: {WEATHER_CODE_TABLE[data["current"]["weathercode"]]}")

Adding, Updating, and Deleting Data in APIs

In this part of the demo, you’ll use ReqRes, a REST API designed specifically for testing applications that make calls to an API. You can send GET, POST, PUT, PATCH, and DELETE requests to its endpoints, and it’ll provide a simulated response. As such, it’s perfect for trying out requests other than GET.

Adding a Resource

Pretend that you want to add a new user to the database: Guido van Rossum, the creator of Python. You’ll provide his name and programming language as parameters by making a POST request to create a new resource in the API.

USERS_ENDPOINT = "https://reqres.in/api/users/"
post_data = {
  "language": "Python",
  "creator":  "Guido van Rossum",
}
response = requests.post(USERS_ENDPOINT, data=post_data)

if response.status_code == 201:
  print(f"Successfully POSTED the resource. Status code: {response.status_code}.")
  print("=====")
  data = response.json()
  pretty_json = json.dumps(data, indent=4)
  print(pretty_json)
else:
  print(f"Failed to POST the resource. Status code: {response.status_code}.")

Updating a Resource

There are two ways to update a resource in an API. The first is to make a PUT request, which updates all the data in the resource and requires you to provide all the data.

path_parameter = 123
url = f"{USERS_ENDPOINT}{path_parameter}"
put_data = {
  "language": "Ada",
  "creator":  "Jean Ichbiah",
}
response = requests.put(url, data=put_data)

if response.status_code == 200:
  print(f"Successfully PUT the resource. Status code: {response.status_code}.")
  print("=====")
  data = response.json()
  pretty_json = json.dumps(data, indent=4)
  print(pretty_json)
else:
  print(f"Failed to PUT the resource. Status code: {response.status_code}.")

path_parameter = 123
url = f"{USERS_ENDPOINT}{path_parameter}"
patch_data = {
  "language": "Ada (named after Ada Lovelace)",
}
response = requests.put(url, data=patch_data)

if response.status_code == 200:
  print(f"Successfully PATCHed the resource. Status code: {response.status_code}.")
  print("=====")
  data = response.json()
  pretty_json = json.dumps(data, indent=4)
  print(pretty_json)
else:
  print(f"Failed to PATCH the resource. Status code: {response.status_code}.")

Deleting a Resource

Now, delete the resource with the ID of 123 using the DELETE request. Run the following in a new code cell to do this:

path_parameter = 123
url = f"{USERS_ENDPOINT}{path_parameter}"
response = requests.delete(url)

if response.status_code == 204:
  print(f"Successfully DELETEd the resource. Status code: {response.status_code}.")
else:
  print(f"Failed to DELETE the resource. Status code: {response.status_code}.")
See forum comments
Cinema mode Download course materials from Github
Previous: Getting Online Data Next: Conclusion