Skip to content

Lesson 4 of 7

Insecure deserialisation

Listen to an AI summary of Insecure deserialisation

0:00 / 0:41

What is an object

An object is a bundle of related values that a program keeps together. The shop keeps each customer's basket as an object with two values, named item and quantity.

What is serialisation

An object exists only inside a running program. Serialisation is turning an object into text or bytes (small numbers) so that the object can be stored or sent. Deserialisation is turning the text or bytes back into an object.

Saving and loading a game

A game saves a player's progress in a save file and loads the save file the next day. Saving is serialisation, the save file is the text or bytes, and loading is deserialisation.

A player can edit the save file and change 10 coins to 9999. The game loads whatever the save file holds. A program that deserialises data builds whatever the data describes.

Two formats

A format is a set of rules for how data is written. JSON is a text format for data, written as names and values inside curly brackets.

Pickle is a format of bytes, read and written by pickle, a Python library.

  1. item: hoodie quantity: 2the basket as an object inside the program
  2. {"item": "hoodie", "quantity": 2}the basket serialised as JSON, which is text
  3. 80 04 95 22 00 ... 68 6f 6f 64 69 65 ... 4b 02 75 2ethe basket serialised as pickle, which is bytes (each pair of characters is one byte)
One basket shown three ways, with some of the bytes left out.

The basket in a cookie

The sports shop stores each customer's basket in a cookie. The browser sends the cookie back with every later request.

The shop serialises the basket with pickle and writes the bytes into the cookie as letters and digits. The Python code below runs on the server when a request arrives. Look for the line that deserialises the cookie:

  1. text = request.cookies["basket"]# reads the cookie from the request
  2. data = base64.b64decode(text)# turns the letters and digits back into bytes
  3. basket = pickle.loads(data)# the function pickle.loads deserialises the bytes into a basket
The shop's code that reads the basket.

A cookie written by an attacker

The browser stores the cookie, so the person who uses the browser can change the cookie. A cookie is untrusted data.

A save file holds only data, and the comparison with a game stops there. Pickle can also name a Python function, and pickle.loads runs the function while loading. The function os.system runs a command, a typed instruction that tells a computer to run a program.

  1. cosc means find a function, in the part of Python named os
  2. systemthe name of the function, so the function is os.system
  3. (S'rm orders.db'the text to pass to the function
  4. tR.R means run the function now
A pickle that runs rm orders.db, a command that deletes the file of the shop's orders.

The attacker writes the pickle as letters and digits, puts the result in the cookie named basket and sends a request. Line 3 of the shop's code deserialises the cookie, and pickle.loads runs rm orders.db on the server. The command runs during line 3, so a check after line 3 is too late.

What is insecure deserialisation

Insecure deserialisation is deserialising untrusted data in a format that can make the program run code while the object is rebuilt. CWE names the weakness deserialisation of untrusted data (CWE-502).

The same basket as JSON

JSON holds only data, such as text, numbers and lists. JSON cannot name a function. The Python function json.loads deserialises JSON and runs no code from the data. Compare the two versions:

# Weakness: pickle can run a function while the cookie loads
basket = pickle.loads(base64.b64decode(request.cookies["basket"]))

# Safer: JSON holds only data, and json.loads runs no code from the cookie
basket = json.loads(base64.b64decode(request.cookies["basket"]))
# The cookie is still untrusted data, so check every value before use

The attacker's pickle is not JSON, so json.loads stops with an error.

  1. cookie -> pickle.loads -> runs rm orders.dbpickle can run code while loading
  2. cookie -> json.loads -> stops with an errorJSON holds only data
The attacker's cookie through each format.

An attacker can still write a JSON basket with a quantity of -5. The shop checks that quantity is a number from 1 to 10.

Prototype pollution

JSON holds only data, and the next weakness is in what code does with the data. In JavaScript, every object takes some values from one shared object, called the prototype. The name __proto__ on any object stands for the prototype. A value written to the prototype shows up on every object.

A deep merge copies every name from one object into another, level by level. The sports shop merges the JSON body of a request into the settings of a customer. Look at line 4.

  1. function merge(target, source) {
  2. for (const key in source) {// every name in the JSON, whatever the name is
  3. if (typeof source[key] === 'object') {
  4. merge(target[key], source[key])// goes one level down
  5. } else {
  6. target[key] = source[key]// copies the value
  7. }
  8. }
  9. }
  10. merge(customer.settings, req.body)// the JSON comes from the request
A deep merge of data from a request.

An attacker sends the JSON {"__proto__": {"isAdmin": true}}. On line 4, target[key] is the prototype, so the merge writes isAdmin to the prototype. Every object in the application now has isAdmin set to true. The names constructor and prototype, used together, lead to the same shared object.

Untrusted data that changes the prototype is prototype pollution (CWE-1321). On a server, a polluted value can reach code that starts a program, and the attacker then runs a command. Safer code copies only the names on an allowlist, and refuses the names __proto__, constructor and prototype.

Examples of fixes

  • JSON, or another format that holds only data, with validation of every value after loading.
  • A library's safer function, such as yaml.safe_load in Python. YAML is another text format for data.
  • A fixed list of the kinds of object the data may build (an allowlist), such as ObjectInputFilter in Java.

Examples of formats that can run code

Each row gives names, from one language, that can run code while deserialising:

Python
pickle, the library jsonpickle, and yaml.load in the library PyYAML with any loader other than SafeLoader, which reads data only.
Java
ObjectInputStream, readObject, XMLDecoder and the library XStream.
C#, which runs on software named .NET
BinaryFormatter, which stays weak whatever its settings. From .NET 9, BinaryFormatter always stops with an error. System.Text.Json reads JSON as data only.
PHP
unserialize.
Ruby
Marshal.load.
JavaScript on a server
unserialize in the library node-serialize.
JSON with type names switched on
A setting that lets JSON name the kind of object to build. The setting is named default typing in the Java library Jackson and TypeNameHandling in the C# library Json.NET.

Examples of what to look for

  • A new line that deserialises data from a request, such as pickle.loads( or yaml.load( on a cookie.
  • A name from the table used on data from a form, an uploaded file or another service.
  • A setting that switches on type names in a JSON or YAML library, such as TypeNameHandling.
  • A deep merge or a copy of every name from JSON in a request into an object, in JavaScript.
  • A merge with no check for the names __proto__, constructor and prototype.

Example grep searches for Python, Java, PHP and Ruby:

Terminal
# Each search is a command, typed into a terminal (a window for typing commands).
# The options -rn search every file in the folder shop/ and print each line number.
# Python
grep -rn "pickle.load" shop/
grep -rn "yaml.load" shop/
# Java
grep -rn "readObject" shop/
# PHP
grep -rn "unserialize" shop/
# Ruby
grep -rn "Marshal.load" shop/

Check your understanding

  1. State what serialisation and deserialisation do.

    Show the answer

    Serialisation turns an object into text or bytes. Deserialisation turns the text or bytes back into an object.

  2. Identify the line of the sports shop's code that deserialises the cookie.

    Show the answer

    Line 3, basket = pickle.loads(data).

  3. Give one reason why JSON is a safer format than pickle for data from a request.

    Show the answer

    JSON holds only data and cannot name a function, so json.loads runs no code from the data.

Practise it

Challenges with a weakness this lesson teaches.