Sum intensity analysis¶

Sum intensity analysis measures normalized line intensity along a kymograph ROI and detects calcium transient peaks. By default, onsets are detected with derivative threshold on the time derivative of df/f0; refined peaks are located afterward in a search window around each onset.

This notebook walks through:

Load an AcqImageList from a local folder.
Select the first recording and add a rectangular ROI.
Inspect detection parameters and choose a built-in preset (fast, medium, or slow).
Run sum-intensity analysis on that ROI.
Plot the detection derivative with its threshold, then df/f0 with peak markers.

Algorithm details (filtering, detrending, thresholding) are covered in the companion notebook sum-intensity-algorithm.ipynb.

Where this fits in CloudScope¶

This notebook uses the acqstore scripting API directly. In everyday use you would tune detection parameters interactively in the CloudScope GUI (left toolbar) on a representative file, then apply a fixed parameter set across many files with AcqImageList.

The plot traces and event markers here mirror what SumIntensityPlotView displays in the app.

In [48]:

Copied!





import numpy as np
import pandas as pd
import plotly.graph_objects as go
import plotly.io as pio

pio.renderers.default = "notebook"

from acqstore.acq_image import AcqImageList
from acqstore.acq_image.analysis.sum_intensity_analysis.sum_intensity_analysis import (
    SumIntensityAnalysis,
)
from acqstore.acq_image.analysis.sum_intensity_analysis.sum_intensity_core import (
    PeakWidthLevel,
    SumIntensityEventPointKey,
    SumIntensityTraceKey,
)
from acqstore.acq_image.analysis.sum_intensity_analysis.sum_intensity_presets import (
    SumIntensityPresetName,
)
import numpy as np
import pandas as pd
import plotly.graph_objects as go
import plotly.io as pio

pio.renderers.default = "notebook"

from acqstore.acq_image import AcqImageList
from acqstore.acq_image.analysis.sum_intensity_analysis.sum_intensity_analysis import (
    SumIntensityAnalysis,
)
from acqstore.acq_image.analysis.sum_intensity_analysis.sum_intensity_core import (
    PeakWidthLevel,
    SumIntensityEventPointKey,
    SumIntensityTraceKey,
)
from acqstore.acq_image.analysis.sum_intensity_analysis.sum_intensity_presets import (
    SumIntensityPresetName,
)

Load recordings from a folder¶

We load the first file from a local Control dataset folder. Edit DATA_FOLDER when adapting this notebook to your own data.

In [49]:

Copied!





DATA_FOLDER = "/Users/cudmore/Dropbox/data/rabbit-ca-variance/raw-data/jan-12-2022/Control"

acq_list = AcqImageList(DATA_FOLDER)
acq = acq_list.get_files()[1]

channel_indices = acq.images.channel_indices
CHANNEL = channel_indices[0]

print("file:", acq.name)
print("path:", acq.path)
print("channels:", channel_indices)
DATA_FOLDER = "/Users/cudmore/Dropbox/data/rabbit-ca-variance/raw-data/jan-12-2022/Control"

acq_list = AcqImageList(DATA_FOLDER)
acq = acq_list.get_files()[1]

channel_indices = acq.images.channel_indices
CHANNEL = channel_indices[0]

print("file:", acq.name)
print("path:", acq.path)
print("channels:", channel_indices)

file: 220110n_0005.tif
path: /Users/cudmore/Dropbox/data/rabbit-ca-variance/raw-data/jan-12-2022/Control/220110n_0005.tif.frames/220110n_0005.tif
channels: [0]

Add an ROI¶

Sum-intensity analysis runs on a rectangular ROI crop. Here we create one ROI spanning the full image bounds (the default when no bounds are supplied).

In [50]:

Copied!

roi = acq.rois.create_rect_roi(name="sum_intensity_notebook", note="docs notebook ROI")
ROI_ID = roi.roi_id

print("roi_id:", ROI_ID, "bounds:", roi.bounds)
roi = acq.rois.create_rect_roi(name="sum_intensity_notebook", note="docs notebook ROI")
ROI_ID = roi.roi_id

print("roi_id:", ROI_ID, "bounds:", roi.bounds)

roi_id: 1 bounds: RectRoiBounds(dim0_start=0, dim0_stop=2500, dim1_start=0, dim1_stop=332)

Detection parameters¶

Sum-intensity detection defaults to derivative_threshold: the detector compares the time derivative of the selected detection source (default df/f0) against derivative_threshold_per_sec. Accepted onsets are refined into peaks within peak_search_window_ms.

Built-in presets (fast, medium, slow) supply complete detection-parameter dictionaries with different refractory and search-window timings. Pick one below, then inspect the full schema table. Edit PRESET_NAME to match your data kinetics.

In [51]:

Copied!





# Edit this: fast, medium, or slow
PRESET_NAME = SumIntensityPresetName.SLOW

for preset in SumIntensityAnalysis.get_detection_presets():
    print(f"{preset.name.value:6s}  {preset.display_name}")

detection_params = SumIntensityAnalysis.get_detection_preset_params(PRESET_NAME)

# set derivative_threshold_per_sec to 5
detection_params["derivative_threshold_per_sec"] = 5

print(f"\nUsing preset: {PRESET_NAME.value}")
print(f"  detection_method: {detection_params['detection_method']}")
print(f"  detection_source: {detection_params['detection_source']}")
print(f"  derivative_threshold_per_sec: {detection_params['derivative_threshold_per_sec']}")

SumIntensityAnalysis.get_detection_schema_dataframe()
# Edit this: fast, medium, or slow
PRESET_NAME = SumIntensityPresetName.SLOW

for preset in SumIntensityAnalysis.get_detection_presets():
    print(f"{preset.name.value:6s}  {preset.display_name}")

detection_params = SumIntensityAnalysis.get_detection_preset_params(PRESET_NAME)

# set derivative_threshold_per_sec to 5
detection_params["derivative_threshold_per_sec"] = 5

print(f"\nUsing preset: {PRESET_NAME.value}")
print(f"  detection_method: {detection_params['detection_method']}")
print(f"  detection_source: {detection_params['detection_source']}")
print(f"  derivative_threshold_per_sec: {detection_params['derivative_threshold_per_sec']}")

SumIntensityAnalysis.get_detection_schema_dataframe()

fast    Fast events
medium  Medium events
slow    Slow events

Using preset: slow
  detection_method: derivative_threshold
  detection_source: df_f_signal
  derivative_threshold_per_sec: 5

Out[51]:

	display_name	type	default	choices	unit	editable	visible	methods	category	description
name
window_radius_points	Window Radius	int	0	None	points	True	True	None	Preprocessing	Radius around each time row used for rolling r...
filter_method	Filter Method	enum	median	(none, median)	NaN	True	True	None	Preprocessing	Optional pre-detection trace filter applied to...
median_filter_kernel_points	Median Filter Kernel	int	3	None	points	True	True	(median,)	Preprocessing	Median filter kernel size in time points. Even...
detrend_method	Detrend Method	enum	single_exponential	(none, single_exponential)	NaN	True	True	None	Preprocessing	Optional bleach-trend removal before detection.
baseline_method	F0 Baseline Method	enum	percentile	(percentile, manual)	NaN	True	True	None	Preprocessing	Method used to estimate scalar F0 for delta-F ...
baseline_percentile	F0 Baseline Percentile	float	20.0	None	percentile	True	True	(percentile,)	Preprocessing	Percentile of the filtered and detrended trace...
manual_f0_baseline	Manual F0 Baseline	float	1.0	None	NaN	True	True	(manual,)	Preprocessing	User-supplied scalar F0 used when baseline_met...
baseline_min_value	F0 Minimum Value	float	0.0	None	NaN	True	False	None	Preprocessing	Small positive floor used to avoid division by...
detection_method	Detection Method	enum	derivative_threshold	(derivative_threshold, absolute_threshold)	NaN	True	True	None	Peak Detection	Peak onset detector.
polarity	Polarity	enum	positive	(positive, negative)	NaN	True	True	None	Peak Detection	Expected peak polarity in the detection signal.
detection_source	Detection Source	enum	df_f_signal	(sum_intensity, norm_sum_intensity, filtered_n...	NaN	True	True	None	Peak Detection	Continuous trace used for onset detection. Der...
absolute_threshold	Absolute Threshold	float	0.0	None	NaN	True	True	(absolute_threshold,)	Peak Detection	Detection-signal threshold used by absolute_th...
derivative_threshold_per_sec	Derivative Threshold	float	1.0	None	1/s	True	True	(derivative_threshold,)	Peak Detection	Derivative threshold in selected detection-sou...
refractory_period_ms	Refractory Period	float	10.0	None	ms	True	True	None	Peak Detection	Minimum accepted onset-to-onset interval.
peak_search_window_ms	Peak Search Window	float	50.0	None	ms	True	True	None	Peak Detection	Forward search window used to refine peak inde...
width_search_window_ms	Width Search Window	float	750.0	None	ms	True	True	None	Peak Detection	Maximum forward search from the refined peak t...
level_fractions	Level Fractions	str	0.1,0.2,0.5,0.8,0.9	None	NaN	True	False	None	Peak Detection	Comma-separated peak-amplitude fractions for w...

Run sum-intensity analysis¶

We run analysis with the selected preset via create_and_run(). On large kymographs this can take a few seconds. Progress is reported through the optional AnalysisRunContext.

In [52]:

Copied!





from acqstore.acq_image.analysis.model import AnalysisRunContext

context = AnalysisRunContext(
    progress_callback=lambda fraction, message: print(f"  progress={fraction:.2f}: {message}")
)

sum_intensity = acq.analysis_set.create_and_run(
    SumIntensityAnalysis,
    channel=CHANNEL,
    roi_id=ROI_ID,
    detection_params=detection_params,
    replace_existing=True,
    context=context,
)

print("summary:")
for key, value in sum_intensity.result.summary.items():
    if key == "peak_events":
        print(f"  {key}: {len(value)} events")
        continue
    print(f"  {key}: {value}")
from acqstore.acq_image.analysis.model import AnalysisRunContext

context = AnalysisRunContext(
    progress_callback=lambda fraction, message: print(f"  progress={fraction:.2f}: {message}")
)

sum_intensity = acq.analysis_set.create_and_run(
    SumIntensityAnalysis,
    channel=CHANNEL,
    roi_id=ROI_ID,
    detection_params=detection_params,
    replace_existing=True,
    context=context,
)

print("summary:")
for key, value in sum_intensity.result.summary.items():
    if key == "peak_events":
        print(f"  {key}: {len(value)} events")
        continue
    print(f"  {key}: {value}")

  progress=0.00: Loading ROI image
  progress=0.25: Running sum intensity analysis
  progress=1.00: Sum intensity analysis complete
summary:
  analysis_date: 260629
  analysis_time: 00:12:38.302
  analysis_version: 1
  status: ok
  num_timepoints: 2500
  num_peaks: 6
  num_space_pixels: 332
  seconds_per_line: 0.0044284
  f0_baseline: 672.9184093831806
  baseline_method: percentile
  baseline_percentile: 20.0
  manual_f0_baseline: 1.0
  peak_amplitude_mean: 0.9483794248294911
  peak_amplitude_median: 0.9495171166424798
  warnings: []
  errors: []
  detrend_method: single_exponential
  detection_method: derivative_threshold
  detection_source: df_f_signal
  peak_search_window_ms: 250.0
  width_search_window_ms: 750.0
  peak_events: 6 events

/Users/cudmore/Sites/cloudscope/src/acqstore/acq_image/analysis/sum_intensity_analysis/sum_intensity_core.py:831: RuntimeWarning: overflow encountered in exp
  return a * np.exp(-b * time_sec) + c
/Users/cudmore/Sites/cloudscope/src/acqstore/acq_image/analysis/sum_intensity_analysis/sum_intensity_core.py:831: RuntimeWarning: overflow encountered in multiply
  return a * np.exp(-b * time_sec) + c

Plot detection derivative and threshold¶

With derivative_threshold detection, the threshold is a horizontal line on the derivative of df/f0 (d_df_f_signal, units 1/s). Onsets mark where the derivative crosses this threshold.

In [53]:

Copied!





deriv = sum_intensity.get_trace(SumIntensityTraceKey.D_DF_F_SIGNAL)
params = sum_intensity.detection_params
threshold = float(params["derivative_threshold_per_sec"])

fig_deriv = go.Figure()
fig_deriv.add_trace(
    go.Scatter(
        x=deriv.x,
        y=deriv.y,
        mode="lines",
        name=deriv.name,
        line={"color": "#1f77b4"},
    )
)
fig_deriv.add_trace(
    go.Scatter(
        x=deriv.x,
        y=np.full(len(deriv.x), threshold),
        mode="lines",
        name=f"Threshold ({threshold:g} 1/s)",
        line={"color": "#d62728", "dash": "dash"},
    )
)

fig_deriv.update_layout(
    title=f"Detection derivative - {acq.name} ch={CHANNEL} roi={ROI_ID}",
    xaxis_title=deriv.x_label,
    yaxis_title=deriv.y_label,
    legend={"orientation": "h", "yanchor": "top", "y": -0.2, "x": 0.5, "xanchor": "center"},
    margin={"l": 60, "r": 20, "t": 50, "b": 80},
)
fig_deriv.show()
deriv = sum_intensity.get_trace(SumIntensityTraceKey.D_DF_F_SIGNAL)
params = sum_intensity.detection_params
threshold = float(params["derivative_threshold_per_sec"])

fig_deriv = go.Figure()
fig_deriv.add_trace(
    go.Scatter(
        x=deriv.x,
        y=deriv.y,
        mode="lines",
        name=deriv.name,
        line={"color": "#1f77b4"},
    )
)
fig_deriv.add_trace(
    go.Scatter(
        x=deriv.x,
        y=np.full(len(deriv.x), threshold),
        mode="lines",
        name=f"Threshold ({threshold:g} 1/s)",
        line={"color": "#d62728", "dash": "dash"},
    )
)

fig_deriv.update_layout(
    title=f"Detection derivative - {acq.name} ch={CHANNEL} roi={ROI_ID}",
    xaxis_title=deriv.x_label,
    yaxis_title=deriv.y_label,
    legend={"orientation": "h", "yanchor": "top", "y": -0.2, "x": 0.5, "xanchor": "center"},
    margin={"l": 60, "r": 20, "t": 50, "b": 80},
)
fig_deriv.show()

Plot df/f0 with threshold crossings and peaks¶

After derivative-threshold detection, onsets mark accepted threshold crossings projected onto df/f0. Peaks are refined maxima in the search window after each onset. Use distinct marker colors to compare timing and amplitude.

In [54]:

Copied!





df_f = sum_intensity.get_trace(SumIntensityTraceKey.DF_F_SIGNAL)
onsets = sum_intensity.get_event_points(SumIntensityEventPointKey.ONSETS)
peaks = sum_intensity.get_event_points(SumIntensityEventPointKey.PEAKS)
width_50 = sum_intensity.get_width_trace(PeakWidthLevel.WIDTH_50)

fig = go.Figure()
fig.add_trace(
    go.Scatter(
        x=df_f.x,
        y=df_f.y,
        mode="lines",
        name=df_f.name,
        line={"color": "#1f77b4"},
    )
)
fig.add_trace(
    go.Scatter(
        x=width_50.x,
        y=width_50.y,
        mode="lines",
        name="HW 0.5",
        line={"color": "#9467bd", "width": 2},
        connectgaps=False,
    )
)
fig.add_trace(
    go.Scatter(
        x=onsets.x,
        y=onsets.y,
        mode="markers",
        name="Threshold crossing (onset)",
        marker={"size": 8, "color": "#d62728", "symbol": "triangle-up"},
    )
)
fig.add_trace(
    go.Scatter(
        x=peaks.x,
        y=peaks.y,
        mode="markers",
        name="Peaks (amplitude)",
        marker={"size": 8, "color": "#2ca02c", "symbol": "circle-open"},
    )
)

f0 = sum_intensity.result.summary.get("f0_baseline")
num_peaks = sum_intensity.result.summary.get("num_peaks")
fig.update_layout(
    title=f"Sum intensity df/f0 - {acq.name} ch={CHANNEL} roi={ROI_ID} ({num_peaks} peaks)",
    xaxis_title=df_f.x_label,
    yaxis_title=df_f.y_label,
    legend={"orientation": "h", "yanchor": "top", "y": -0.15, "x": 0.5, "xanchor": "center"},
    margin={"l": 60, "r": 20, "t": 50, "b": 70},
    annotations=[
        {
            "text": f"F0={float(f0):.4g}" if f0 is not None else "F0 unavailable",
            "xref": "paper",
            "yref": "paper",
            "x": 0.01,
            "y": 0.99,
            "showarrow": False,
            "align": "left",
        }
    ],
)
fig.show()
df_f = sum_intensity.get_trace(SumIntensityTraceKey.DF_F_SIGNAL)
onsets = sum_intensity.get_event_points(SumIntensityEventPointKey.ONSETS)
peaks = sum_intensity.get_event_points(SumIntensityEventPointKey.PEAKS)
width_50 = sum_intensity.get_width_trace(PeakWidthLevel.WIDTH_50)

fig = go.Figure()
fig.add_trace(
    go.Scatter(
        x=df_f.x,
        y=df_f.y,
        mode="lines",
        name=df_f.name,
        line={"color": "#1f77b4"},
    )
)
fig.add_trace(
    go.Scatter(
        x=width_50.x,
        y=width_50.y,
        mode="lines",
        name="HW 0.5",
        line={"color": "#9467bd", "width": 2},
        connectgaps=False,
    )
)
fig.add_trace(
    go.Scatter(
        x=onsets.x,
        y=onsets.y,
        mode="markers",
        name="Threshold crossing (onset)",
        marker={"size": 8, "color": "#d62728", "symbol": "triangle-up"},
    )
)
fig.add_trace(
    go.Scatter(
        x=peaks.x,
        y=peaks.y,
        mode="markers",
        name="Peaks (amplitude)",
        marker={"size": 8, "color": "#2ca02c", "symbol": "circle-open"},
    )
)

f0 = sum_intensity.result.summary.get("f0_baseline")
num_peaks = sum_intensity.result.summary.get("num_peaks")
fig.update_layout(
    title=f"Sum intensity df/f0 - {acq.name} ch={CHANNEL} roi={ROI_ID} ({num_peaks} peaks)",
    xaxis_title=df_f.x_label,
    yaxis_title=df_f.y_label,
    legend={"orientation": "h", "yanchor": "top", "y": -0.15, "x": 0.5, "xanchor": "center"},
    margin={"l": 60, "r": 20, "t": 50, "b": 70},
    annotations=[
        {
            "text": f"F0={float(f0):.4g}" if f0 is not None else "F0 unavailable",
            "xref": "paper",
            "yref": "paper",
            "x": 0.01,
            "y": 0.99,
            "showarrow": False,
            "align": "left",
        }
    ],
)
fig.show()

Inspect results¶

The result table stores continuous traces as columns (time_sec, df_f_signal, d_df_f_signal, ...). Peak metadata is also serialized in the summary under peak_events.

In [55]:

Copied!

cols = ["time_sec", "df_f_signal", "d_df_f_signal", "is_onset", "is_peak"]
sum_intensity.result.table[cols].head(10)
cols = ["time_sec", "df_f_signal", "d_df_f_signal", "is_onset", "is_peak"]
sum_intensity.result.table[cols].head(10)

Out[55]:

	time_sec	df_f_signal	d_df_f_signal	is_onset	is_peak
0	0.000000	0.027121	0.012893	False	False
1	0.004428	0.027178	-0.104356	False	False
2	0.008857	0.026197	-0.498051	False	False
3	0.013285	0.022767	-0.264563	False	False
4	0.017714	0.023854	0.129132	False	False
5	0.022142	0.023911	0.012893	False	False
6	0.026570	0.023968	-0.266585	False	False
7	0.030999	0.021550	-0.266585	False	False
8	0.035427	0.021607	0.012893	False	False
9	0.039856	0.021664	0.468751	False	False

Takeaways¶

Sum-intensity analysis produces a df/f0 time series and detected peak events on one rectangular ROI.
Default detection uses derivative threshold on df/f0; tune derivative_threshold_per_sec and kinetic windows via presets or the schema.
Start from a built-in preset (fast, medium, slow), then edit parameters in the CloudScope GUI or notebook before batch runs.
Plot the derivative + threshold to QC onset detection, then df/f0 + peaks for amplitude review.
See sum-intensity-algorithm.ipynb for detector internals.