Sans
- Sep 27, 2022
- 2 min read

How to collect Network Logs using WebRequest API (Chrome Extension Manifest v3)

Introduction

Devign provides the essential context and data for developers such as console logs, network logs, dev environment, user flow, user footprints, and more—to resolve the issue.

This article shines light upon how the Devign fetches the network log information using Chrome’s webRequest API (HAR - HTTP Archive) which is useful for debugging.

⚠ Warning

HAR logs capture all network information including personal information such as cookies or passwords. Be careful about whom you share the log information with!

How the Chrome Extension webRequest API works

Manifest

Your manifest.json must include webRequest permission in order to access webRequests API

Side Note

The web request API presents an abstraction of the network stack to the extension.

Internally, one URL request can be split into several HTTP requests (for example to fetch individual byte ranges from a large file) or can be handled by the network stack without communicating with the network.

Background (in version 3, referred to as service workers)

What service workers do: listen for and respond to events in order to enhance the end user's experience.

webRequest API events used:

onBeforeSendHeaders

Fired before sending an HTTP request, once the request headers are available. This may occur after a TCP connection is made to the server, but before any HTTP data is sent.

chrome.webRequest.onBeforeSendHeaders.addListener(
  callback: function,
  filter: RequestFilter,
  extraInfoSpec?: OnBeforeSendHeadersOptions[],
)

onHeadersReceived

Fired when HTTP response headers of a request have been received.

chrome.webRequest.onHeadersReceived.addListener(
  callback: function,
  filter: RequestFilter,
  extraInfoSpec?: OnHeadersReceivedOptions[],
)

src/background/index.ts

Note: The following example stores the network log data in an array but Devign stores the data in a queue.

Ignoring requests initiated by the extension with :if(details.initiator?.includes('chrome-extension://')) return;
Storing requestTimeStamp and responseTimeStamp separately
Example of the Request, Response Header structures (Variable - depends on the individual request type)

Request Headers & Response Headers

src/background/api.ts - Post the collected network log data to backend server

Content Scripts

This is the part of the extension which can access the Document Object Model (DOM) of the webpage and tells the service workers when to start collecting the network log data.

Devign extension uses a toggle button to decide whether to collect logs or not. When toggled on, the content script sends a message to the background to capture the network logs using the following code: