
Description
Continuously profile applications with Grafana Pyroscope and read the result as flame graphs. Covers three instrumentation paths — language SDK push (Go / Java / Python / Ruby / Node / .NET / Rust), Alloy eBPF auto-instrumentation (no code change, requires kernel 5.8+ with BTF), and SDK → Alloy receiver — plus ProfileQL queries, profile types (CPU / memory / allocations / goroutines / mutex), Grafana Cloud Profiles endpoint, and Span Profiles trace-to-profile linking. Use when adding profiling to a service, deploying Alloy as a cluster-wide eBPF profiler, hunting CPU / memory hotspots from a flame graph, comparing two profiles to find a regression, or correlating a slow Tempo trace to its profile — even when the user says "find what's burning CPU", "flame graph this app", "continuous profiling", "heap hotspots", or "why is allocation so high" without naming Pyroscope.
SKILL.md
Grafana Pyroscope
Continuous profiling — flame graphs of CPU, memory, allocations, mutex contention, goroutines.
Prerequisites
- Pyroscope server (OSS) or Grafana Cloud Profiles endpoint
- For Cloud: numeric Pyroscope user (stack id) + API key
- For eBPF via Alloy: root + host PID + Linux ≥ 5.8 with BTF (or RHEL 4.18+)
Instrumentation paths
- Alloy eBPF (preferred) — auto-instrument, no code change
- SDK direct push — application calls Pyroscope API
- SDK → Alloy — SDK posts to
pyroscope.receive_http, Alloy forwards
Common Workflows
1. Instrument an app with the SDK (representative: Python)
pip install pyroscope-io==1.0.11
import pyroscope, os
pyroscope.configure(
application_name="my.python.app",
server_address="http://pyroscope:4040",
sample_rate=100, oncpu=True,
tags={"region": os.getenv("REGION"), "env": "prod"},
)
# Dynamic tag for a hot path
with pyroscope.tag_wrapper({"controller": "slow_controller"}):
slow_code()
# Verify the app is pushing — Pyroscope ingests samples in a few seconds
curl -s http://pyroscope:4040/ready # → "ready"
curl -s http://pyroscope:4040/api/v1/labels | jq '.data | index("service_name")' # → not null
# Verify the service shows up in Grafana → Explore → Profiles → service dropdown.
Other SDKs (Java agent, Node, Ruby, .NET, Rust) + Cloud auth + tunable env vars: references/sdks.md.
2. Cluster-wide eBPF profiling with Alloy
# config.alloy — full block in references/ebpf-and-query.md
pyroscope.ebpf "local_pods" {
forward_to = [pyroscope.write.cloud.receiver]
targets = discovery.relabel.local_pods.output
sample_rate = 97
collect_interval = "15s"
}
pyroscope.write "cloud" {
endpoint {
url = "https://profiles-prod-xxx.grafana.net"
basic_auth { username = sys.env("PYROSCOPE_USER")
password = sys.env("GRAFANA_API_KEY") }
}
}
# 1. Reload Alloy
curl -X POST http://localhost:12345/-/reload
# 2. Verify the eBPF component is healthy
curl -s http://localhost:12345/api/v0/web/components \
| jq '.[] | select(.id|contains("pyroscope.ebpf")) | {id,health:.health.state}'
# Expect: health.state == "healthy"
# 3. Verify profiles arriving in Pyroscope
# Grafana → Explore → Profiles datasource → query:
# {namespace="default", __profile_type__="process_cpu:cpu:nanoseconds:cpu:nanoseconds"}
# Expect flame graph to render with frames from the target pods.
3. Query with ProfileQL
{service_name="myapp", env="prod",
__profile_type__="process_cpu:cpu:nanoseconds:cpu:nanoseconds"}
Profile-type list + full ProfileQL grammar: references/ebpf-and-query.md.
Troubleshooting
- SDK starts but no flame graph → check the app actually called
start()/configure()(some SDKs are lazy); checkserver_addressreachable from inside the container - Alloy eBPF component
unhealthywith BPF errors → kernel < 5.8 or BTF missing;ls /sys/kernel/btf/vmlinux - Cloud push 401 → wrong
basic_auth_username(must be the numeric stack id, not the slug) - Profile shows up but with no frames → for Java, set
PYROSCOPE_FORMAT=jfr; for Python on Alpine, ensureprocfsandglibccompatibility
Resources
- Pyroscope docs
- Grafana Cloud Profiles
references/sdks.md— Java/Node/Ruby/.NET/Rust install + config + env-var table + profile-type matrixreferences/ebpf-and-query.md— full Alloy eBPF pipeline + ProfileQL
More skills from the skills repository
View all 48 skillsadaptive-metrics
optimize Grafana Cloud metrics costs
Jul 12Cost OptimizationGrafanaMetricsObservabilityadmin
manage Grafana Cloud accounts and RBAC
Jul 12Access ControlAuthGrafanaOperationsadmission-control
implement admission control webhooks
Jul 12ArchitectureSecurityValidationalerting-irm
configure Grafana Alerting and Incident Management
Jul 12AlertingGrafanaIncident ResponseMonitoringalloy
build unified telemetry pipelines with Grafana Alloy
Jul 12GrafanaLogsMetricsObservability +1app-observability
monitor application performance in Grafana Cloud
Jul 12APMDistributed TracingGrafanaLLM +1
More from Grafana
View publisherfaro-setup-web
instrument web apps with Grafana Faro
faro-web-sdk
Jul 12Distributed TracingFrontendMonitoringObservabilityconfiguring-yesoreyeram-infinity-datasource
configure Grafana Infinity data source
grafana-infinity-datasource
Jul 12API DevelopmentAuthenticationConfigurationGrafanaquerying-yesoreyeram-infinity-datasource
query data with Infinity datasource
grafana-infinity-datasource
Jul 15CSVData AnalysisGrafanaGraphQL +1agento11y
manage Grafana Agent Observability resources
gcx
Jul 18AgentsGrafanaMonitoringObservabilityagento11y-eval-starter
scaffold evaluation experiments for agents
gcx
Jul 21AgentsEvalsExperimentsGrafanaagento11y-instrument
instrument LLM apps for agent observability
gcx
Jul 21AgentsGrafanaInstrumentationLLM +1