Skip to content

Commit b277c79

Browse files
committed
Major rework, initial commit of topcat_scatter_utils module with core functionality for density calculation and plotting. Added .gitignore, CHANGELOG, and requirements.txt. Implemented calculate_density(), truncate_colormap(), and plot_density_scatter() functions for enhanced scatter plot visualization.
1 parent 202d90b commit b277c79

7 files changed

Lines changed: 498 additions & 68 deletions

File tree

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
__pycache__/
2+
.ipynb_checkpoints/
3+
.DS_Store

CHANGELOG.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# Changelog
2+
3+
## [1.0.0] - 2025-01-XX
4+
5+
### Added
6+
7+
- Major rework
8+
- `calculate_density()` function for KDE-based density calculation
9+
- `truncate_colormap()` function for colormap manipulation
10+
- `plot_density_scatter()` convenience function
11+
- Automatic NaN/inf handling and error warnings
12+
- Comprehensive demo notebook

LICENSE

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
MIT License
22

3-
Copyright (c) 2023 Anna O'Grady
3+
Copyright (c) 2025 Anna O'Grady
44

55
Permission is hereby granted, free of charge, to any person obtaining a copy
66
of this software and associated documentation files (the "Software"), to deal

README.md

Lines changed: 135 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,139 @@
11
# topcat_scatter_python
2-
This repository includes code that will create scatter plots that resemble TOPCAT plots, particularly in how the density map appears.
3-
This code was adapted from the following sources:
4-
https://stackoverflow.com/questions/50526344/points-with-density-gradient
5-
https://stackoverflow.com/questions/20105364/how-can-i-make-a-scatter-plot-colored-by-density-in-matplotlib
62

7-
TOPCAT_density_demo.ipynb contains the code and demonstrates its use.
3+
A set of utilities to create scatter plots in Python that resemble [TOPCAT](http://www.star.bris.ac.uk/~mbt/topcat/) plots. This code was adapted from the these two sources: [Source1](https://stackoverflow.com/questions/50526344/points-with-density-gradient), [Source2](https://stackoverflow.com/questions/20105364/how-can-i-make-a-scatter-plot-colored-by-density-in-matplotlib).
4+
5+
---
6+
7+
## Requirements
8+
9+
* Python 3.8 or higher
10+
* Dependencies below are listed in `requirements.txt`:
11+
* numpy 1.20.0 or higher
12+
* scipy 1.10.0 or higher
13+
* matplotlib 3.3.0 or higher
14+
15+
Install dependencies using pip:
16+
```shell
17+
pip install -r requirements.txt
18+
```
19+
20+
---
21+
22+
## Installation
23+
24+
Clone the repo and install dependencies:
25+
```shell
26+
git clone https://github.com/AnnaOG/topcat_scatter_python.git
27+
cd topcat_scatter_python
28+
pip install -r requirements.txt
29+
```
30+
31+
---
32+
33+
## Features and Comparison
34+
35+
* Calculates point density using Gaussian KDE and sorts points so the densest regions appear on top
36+
* Truncates colormaps to avoid washed-out colors
37+
* Convenience function for quick plotting
38+
* Handles NaN and inf values with warnings
39+
40+
A demo of the features is included in [`TOPCAT_density_demo.ipynb`](./TOPCAT_density_demo.ipynb).
841

942
![Comparison image](topcat_comparison.png)
43+
44+
## Functions Overview
45+
46+
### calculate_density
47+
48+
```python
49+
calculate_density(x, y, bandwidth=None)
50+
```
51+
52+
#### Inputs
53+
54+
* `x, y` (array-like): X/Y coordinates of points
55+
* `bandwidth` (float, optional): Bandwidth for `gaussian_kde`. If None (default), it will be estimated automatically.
56+
57+
#### Returns
58+
59+
* `x_sorted, y_sorted, densities` (array-like): X/Y coordinates and density values, sorted from least to most dense.
60+
* Raises `ValueError` if x and y have different lengths, if there are fewer than 2 points, or if bandwidth is non-positive.
61+
62+
### truncate_colormap
63+
64+
```python
65+
truncate_colormap(cmap, minval=0.4, maxval=0.9, n=256):
66+
```
67+
68+
#### Inputs
69+
70+
* `cmap` (matplotlib.colors.Colormap OR str): The original colormap to be truncated. Can be a colormap object or a string name of a colormap.
71+
* `minval` (float, default=0.4): The minimum value of the colormap to include (between 0 and 1).
72+
* `maxval` (float, default=0.9): The maximum value of the colormap to include (between 0 and 1).
73+
* `n` (int, default=256): The number of discrete colors to generate in the truncated colormap.
74+
75+
#### Returns
76+
77+
* `LinearSegmentedColormap`: The truncated colormap.
78+
* Returns `ValueError` if minval is greater than maxval or if either is outside the allowed range.
79+
80+
### plot_density_scatter
81+
82+
```python
83+
plot_density_scatter(x, y, bandwidth=None, cmap='Reds', minval=0.4, maxval=0.9, n=256, ax=None, **scatter_kwargs):
84+
```
85+
86+
#### Inputs
87+
88+
* `x, y` (array-like): X/Y coordinates of points
89+
* `bandwidth` (float, optional): Bandwidth for `gaussian_kde`. If None (default), it will be estimated automatically.
90+
* `cmap` (matplotlib.colors.Colormap OR str): The original colormap to be truncated. Can be a colormap object or a string name of a colormap.
91+
* `minval` (float, default=0.4): The minimum value of the colormap to include (between 0 and 1).
92+
* `maxval` (float, default=0.9): The maximum value of the colormap to include (between 0 and 1).
93+
* `n` (int, default=256): The number of discrete colors to generate in the truncated colormap.
94+
* `ax` (matplotlib.axes.Axes, optional): Axes object to plot on. If None, uses current axes.
95+
* `**scatter_kwargs` (keyword arguments, optional): Additional keyword arguments passed to plt.scatter().
96+
97+
#### Returns
98+
99+
* `scatter` (matplotlib.collections.PathCollection): The scatter plot object.
100+
101+
---
102+
103+
## Usage Options and Tips
104+
105+
* See [`TOPCAT_density_demo.ipynb`](./TOPCAT_density_demo.ipynb) for full demo
106+
* Leave `bandwidth=None` for automatic estimation (recommended for most cases)
107+
* Adjust `minval`/`maxval` (default 0.4-0.9) to control colormap brightness
108+
* Use `edgecolor='none'` for cleaner appearance (default in convenience function)
109+
110+
### 1. Quick Plot
111+
112+
```python
113+
from topcat_scatter_utils import *
114+
import numpy as np
115+
116+
x = np.random.randn(5000)
117+
y = x * 3 + np.random.randn(5000)
118+
119+
plot_density_scatter(x, y)
120+
```
121+
122+
### 2. Separate Functions (More Control)
123+
124+
```python
125+
from topcat_scatter_utils import *
126+
import numpy as np
127+
import matplotlib.pyplot as plt
128+
129+
x = np.random.randn(5000)
130+
y = x * 3 + np.random.randn(5000)
131+
132+
# Sort data
133+
x_sorted, y_sorted, density = calculate_density(x, y)
134+
# Truncate colormap
135+
cmap_t = truncate_colormap('Purples',0.4,0.9)
136+
137+
# Plot
138+
plt.scatter(x_sorted, y_sorted, c=density, cmap=cmap_t, s=20, edgecolor='None')
139+
```

TOPCAT_density_demo.ipynb

Lines changed: 168 additions & 62 deletions
Large diffs are not rendered by default.

requirements.txt

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Minimum tested versions of dependencies for topcat_scatter_utils
2+
numpy>=1.20.0 # Numerical computing library
3+
scipy>=1.10.0 # Scientific computing library, used for KDE
4+
matplotlib>=3.3.0 # Plotting library

topcat_scatter_utils.py

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
# Requirements
2+
3+
import warnings
4+
import numpy as np
5+
import matplotlib.pyplot as plt
6+
import matplotlib.colors as mcolors
7+
from scipy.stats import gaussian_kde
8+
9+
__all__ = ['calculate_density', 'truncate_colormap', 'plot_density_scatter']
10+
11+
# Density Calculation and Sorting Function
12+
def calculate_density(x, y, bandwidth=None):
13+
"""
14+
Calculate point density for scatter plot data using Gaussian KDE.
15+
16+
Points are sorted by density so that when plotted, denser regions
17+
appear on top (to emulate TOPCAT plotting behaviour).
18+
19+
Parameters:
20+
x : array-like
21+
X coordinates of the points.
22+
y : array-like
23+
Y coordinates of the points.
24+
bandwidth : float, optional
25+
Bandwidth for the KDE. If None, it will be estimated automatically.
26+
27+
Returns:
28+
x_sorted : array-like
29+
X coordinates sorted by density.
30+
y_sorted : array-like
31+
Y coordinates sorted by density.
32+
density: array-like
33+
Density values sorted from least to most dense.
34+
35+
Raises:
36+
ValueError
37+
If x and y have different lengths, if there are fewer than 2 points, or if bandwidth is non-positive.
38+
39+
Example Usage:
40+
x = np.random.randn(100)
41+
y = np.random.randn(100)
42+
x_sorted, y_sorted, density = calculate_density(x, y)
43+
"""
44+
45+
x = np.asarray(x)
46+
y = np.asarray(y)
47+
48+
# Check same length
49+
if len(x) != len(y):
50+
raise ValueError(f"x and y must have same length. Got x: {len(x)}, y: {len(y)}")
51+
52+
# Check minimum points for KDE
53+
if len(x) < 2:
54+
raise ValueError(f"Need at least 2 points for density calculation. Got {len(x)}")
55+
56+
if bandwidth is not None and bandwidth <= 0:
57+
raise ValueError(f"bandwidth must be positive. Got {bandwidth}")
58+
59+
# Handle NaN/inf values
60+
mask = np.isfinite(x) & np.isfinite(y)
61+
if not mask.all():
62+
n_bad = (~mask).sum()
63+
warnings.warn(f"Removing {n_bad} points with NaN or infinite values")
64+
x = x[mask]
65+
y = y[mask]
66+
67+
xy = np.vstack([x, y])
68+
z = gaussian_kde(xy, bw_method=bandwidth)(xy) # Evaluate density
69+
70+
idx = z.argsort()
71+
x_sorted, y_sorted, density = x[idx], y[idx], z[idx] # Sort points by density
72+
73+
return x_sorted, y_sorted, density
74+
75+
def truncate_colormap(cmap, minval=0.4, maxval=0.9, n=256):
76+
"""
77+
Truncate a matplotlib colormap to a specified range.
78+
79+
Used in this script to avoid the light and dark extremes of colormaps
80+
to better emulate the TOPCAT density style.
81+
82+
Parameters:
83+
cmap : matplotlib.colors.Colormap OR str
84+
The original colormap to be truncated. Can be a colormap object or a string name of a colormap.
85+
minval : float, default=0.4
86+
The minimum value of the colormap to include (between 0 and 1).
87+
maxval : float, default=0.9
88+
The maximum value of the colormap to include (between 0 and 1).
89+
n : int, default=256
90+
The number of discrete colors to generate in the truncated colormap.
91+
92+
Returns:
93+
LinearSegmentedColormap
94+
The truncated colormap.
95+
96+
Validates:
97+
- minval must be less than maxval.
98+
- minval and maxval must be between 0 and 1.
99+
100+
Example Usage:
101+
truncated_cmap = truncate_colormap('viridis', 0.2, 0.8)
102+
# or
103+
original_cmap = plt.get_cmap('viridis')
104+
truncated_cmap = truncate_colormap(original_cmap, 0.2, 0.8)
105+
"""
106+
107+
# Allow string input (e.g., 'Reds' instead of plt.get_cmap('Reds'))
108+
if isinstance(cmap, str):
109+
cmap = plt.get_cmap(cmap)
110+
111+
# Validate range
112+
if minval >= maxval:
113+
raise ValueError(f"minval must be less than maxval. Got minval={minval}, maxval={maxval}")
114+
115+
if not (0 <= minval <= 1 and 0 <= maxval <= 1):
116+
raise ValueError(f"minval and maxval must be between 0 and 1. Got minval={minval}, maxval={maxval}")
117+
118+
new_cmap = mcolors.LinearSegmentedColormap.from_list(
119+
'truncated({n},{a:.2f},{b:.2f})'.format(n=cmap.name, a=minval, b=maxval),
120+
cmap(np.linspace(minval, maxval, n))
121+
)
122+
return new_cmap
123+
124+
def plot_density_scatter(x, y, bandwidth=None, cmap='Reds',
125+
minval=0.4, maxval=0.9, n=256, ax=None, **scatter_kwargs):
126+
"""
127+
Create a density scatter plot similar to TOPCAT's density plot style.
128+
129+
This is a convenience function combining calculate_density() and
130+
truncate_colormap(). For more control, use those functions separately.
131+
By default, edgecolor is set to 'none' to match TOPCAT style.
132+
133+
Parameters:
134+
x : array-like
135+
X coordinates of the points.
136+
y : array-like
137+
Y coordinates of the points.
138+
bandwidth : float, optional
139+
Bandwidth for the KDE. If None, it will be estimated automatically.
140+
cmap : str or matplotlib.colors.Colormap, default='Reds'
141+
Colormap to use for density coloring.
142+
minval : float, default=0.4
143+
Minimum value for truncating the colormap.
144+
maxval : float, default=0.9
145+
Maximum value for truncating the colormap.
146+
n : int, default=256
147+
The number of discrete colors to generate in the truncated colormap.
148+
ax : matplotlib.axes.Axes, optional
149+
Axes object to plot on. If None, uses current axes.
150+
**scatter_kwargs : keyword arguments
151+
Additional keyword arguments passed to plt.scatter().
152+
153+
Returns:
154+
scatter : matplotlib.collections.PathCollection
155+
The scatter plot object.
156+
157+
Example Usage:
158+
x = np.random.randn(1000)
159+
y = np.random.randn(1000)
160+
cmap='viridis'
161+
plot_density_scatter(x, y, cmap=cmap, minval=0.2, maxval=0.8)
162+
"""
163+
164+
x_sorted, y_sorted, density = calculate_density(x, y, bandwidth)
165+
truncated_cmap = truncate_colormap(cmap, minval, maxval, n)
166+
167+
if ax is None:
168+
ax = plt.gca()
169+
170+
# Set default edgecolor if not specified by user
171+
scatter_kwargs.setdefault('edgecolor', 'None')
172+
173+
scatter = ax.scatter(x_sorted, y_sorted, c=density,
174+
cmap=truncated_cmap, **scatter_kwargs)
175+
return scatter

0 commit comments

Comments
 (0)