Exporting PxL Scripts to a Plugin

This tutorial shows you how to automatically convert a PxL Script into an OpenTelemetry export script that can be used with a Pixie plugin. Plugin scripts make it easy to extend Pixie's capabilities: you can enable a wide-range of use-cases including long-term storage and control signals for autoscalers.

We released the OpenTelemetry Exporter and Plugin System a few months before, but we got a lot of feedback that writing export scripts was long and repetitive.

We built this new flow to automate out the tedious parts of writing export scripts.

1. You will need a Kubernetes cluster. If you don’t already have one, you can create a minikube cluster following the directions here.

2. You will need to install Pixie on your cluster using one of our install guides.

3. You will need to setup a Pixie plugin. You can setup a simple demo collector to try the feature out.

PxL scripts are used to query telemetry data collected by the Pixie Platform. Here we will write a script that calculates the throughput of requests through each endpoint of each pod in your cluster. In the following section, we'll convert the script into an OpenTelemetry export script.

We'll use the Live UI's Scratch Pad to develop our PxL script.

1. Open Pixie's Live UI.

2. Select the Scratch Pad script from the script drop-down menu in the top left.

3. Open the script editor using the keyboard shortcut: ctrl+e (Windows, Linux) or cmd+e (Mac).

4. Replace the contents of the PxL Script tab with the following.

1import px
2# Read in the http_events table
3df = px.DataFrame(table='http_events', start_time='-10s', end_time=px.now())
4
5# Attach the pod and service metadata
6df.pod = df.ctx['pod']
7df.service = df.ctx['service']
8# Count the number of requests per pod and service
9df = df.groupby(['pod', 'service', 'req_path']).agg(
10  throughput=('latency', px.count),
11  time_=('time_', px.max),
12)
13
14px.display(df, 'http')

5. Run the script using the RUN button in the top right or by using the keyboard shortcut: ctrl+enter (Windows, Linux) or cmd+enter (Mac).

6. Hide the script editor using ctrl+e (Windows, Linux) or cmd+e (Mac).

Your Live UI should output something similar to the following:

This PxL script calculates the throughput of HTTP requests made to each pod in your cluster.

Now that we have our PxL script, let's export it.

7. Open the script editor again (ctrl+e or cmd+e)

8. On the top right corner, click "Export to Plugin". Pixie will process the script and attempt to generate an OpenTelemetry export configuration for it.

   

9. If the export fails, you'll see an error message.

   

10. If the export is successful, you'll be directed to the "Create Export Script" page with the generated OpenTelemetry Export Script already filled in!

    

11. Read over the generated script and make any changes such as adding resource columns or removing undesired metrics. See How it Works for more on how the script is generated.

12. Set the name of the script to my-export-script and set the Plugin provider to OpenTelemetry.

    

13. Click Create and you should see my-export-script in the "Custom Scripts" section. Your data should now be sent to whatever Plugin you configured.

    

When you export a PxL script, we extract the schema generated by the script and auto-generate the OpenTelemetry Export PxL for each table. INT64 and FLOAT64 columns become Gauges while other column types become the resource attributes of the exported Data.

Any calls to px.DataFrame are rewritten to replace the start_time argument with px.plugin.start_time and end_time argument with px.plugin.end_time. These arguments are filled in by the Plugin script-runner with the start and end times of the current summary window. The summary window size can be configured on the "Create Export Script" page.

You've now converted your first PxL script into an OpenTelemetry export script. You can dive deeper into PxL scripts by checking out our "How to Write a PxL Script" series or by reading through the PxL documentation.

You can also learn more about Pixie's OpenTelemetry integration in this tutorial and through the OpenTelemetry Export documentation.

Having problems? Check out the Pixie Plugin Troubleshooting guide.

Export OpenTelemetry DataCollecting Custom Data