CallEnhancer

CallEnhancer restores call-centre speech. It takes narrowband, codec'd, noisy phone audio, such as 8 kHz G.711 or GSM calls, and returns clean speech at 48 kHz.

This is version 2. Version 1 stays available at the v1 tag.

What changed in v2

• The full band. v2's high-band distance to clean speech is 5.5 dB on our 1,289 call-centre clips, against 6.9 dB for v1, and 3.9 dB against 5.3 dB on 58 real calls. Clean studio recordings score 2.0 to 2.3 dB.
• Intelligibility. On the 58 real calls v2 reads 30.8 % nCER and v1 30.1 %. The mean per-call difference is +1.1 points (95 % CI [-0.9, +3.5]), within noise. v2 is better on 27 calls and worse on 30.
• Deletions. v2 deletes less speech. It mutes 10.9 % of speech against 13.2 % for v1, at 40.5 % against 41.9 % silence filled: more than a change in loudness balance explains.
• Two parties in one channel. v2 keeps a quieter second voice. With the two sides of our 58 real calls mixed into one channel, v2 dropped 4.4 % of the quieter party's speech and v1 4.1 %. v2 still drops more than a quarter of it in 4 of 56 calls, v1 in 1.
• Naturalness. About the same. SOTA-MOS is 2.77 for v2 and 2.70 for v1 (300 clips). Two runs of one recipe differ by 0.07 to 0.08.
• Same size, same files. A w2v-BERT 2.0 encoder with a LoRA adapter and a 188M DAC decoder. The weights keep v1's paths, so existing download code keeps working.

Comparison

We ran CallEnhancer v2, CallEnhancer v1 and three open restoration models on the same call-centre audio, through the same scripts.

Each curve is a system's median spectrum, relative to its own 1 to 4 kHz band, on speech-active frames. Black is clean speech. The number in the legend is the high-band distance (HBD): the RMS gap to the black curve from 4 to 20 kHz. Lower is closer to clean speech.

model	output	HBD, 1,289 clips (dB) ↓	HBD, 58 real calls (dB) ↓	nCER, 58 real calls (%) ↓	speech muted (%) ↓	silence filled (%)	SOTA-MOS ↑	DNSMOS overall ↑
CallEnhancer v2 (this release)	48 kHz	5.5	3.9	30.8	10.9	40.5	2.77	2.97
CallEnhancer v1	48 kHz	6.9	5.3	30.1	13.2	41.9	2.70	2.76
sidon-v0.1	48 kHz	4.4	5.2	30.8	28.2	20.9	2.64	3.17
VoiceFixer	44.1 kHz	3.0	2.1	50.0	27.1	52.9	2.19	2.78
Resemble Enhance	44.1 kHz	4.8	8.6	48.8	19.5	15.7	2.28	3.05
input (telephone audio)	8 to 16 kHz	51.3	49.2	48.8

Bold marks the best system in each column. Silence filled has no best (see How we measured).

How we measured

• Test audio. 1,289 call-centre clips (English, Malay and Mandarin, 9.4 hours) and 58 real calls with human transcripts, one party per channel. Both are customer data; we publish aggregates only.
• Systems. Each restores the same files with its default settings. 44.1 kHz outputs are resampled to 48 kHz.
• nCER. Malaysian Whisper against the human transcripts, after LLM normalisation, mean of 3 runs. Each party is restored on its own channel.
• Speech muted. The share of 20 ms speech frames that the output drops by more than 6 dB. A band-pass that deletes nothing scores 5.9 %.
• Silence filled. The share of silent input frames that the output raises by more than 6 dB. A counter-metric, not a score: a change in loudness balance trades one against the other, so compare muting at similar silence filled. The shaded band in the deletions figure is v1 with only its balance changed.
• One channel. Both sides of each real call mixed into one file at their recorded levels (11.4 dB apart at the median). Speech muted is read on the quieter party, at 8 kHz.
• SOTA-MOS (HighRateMOS) on 300 clips at 48 kHz.
• DNSMOS P.835 overall, the median over the clips it can score (1,243 to 1,283 of 1,289).
• Silent outputs. Resemble Enhance returns silence (under -60 dBFS) on 41 clips; the other systems never do. Speech muted counts these as deletions.

Quick start

pip install torch torchaudio "transformers>=4.56" "descript-audio-codec>=1.0.0" soundfile "huggingface_hub>=0.34"

hf download Scicom-intl/CallEnhancer infer_callcentre.py expand_decoder.py \
    fe_callcentre/fe_adapter_full.pt decoder_callcentre/decoder_only.pt --local-dir CallEnhancer
cd CallEnhancer

python infer_callcentre.py --input your_call.wav --out-dir out \
    --fe-adapter fe_callcentre/fe_adapter_full.pt \
    --decoder decoder_callcentre/decoder_only.pt --device cuda

• out/your_call_restored48k.wav is the restored speech at 48 kHz.
• out/your_call_orig48k.wav is the input, upsampled with no model, for an A/B listen.
• --input takes a file or a directory (.wav, .flac, .mp3, .ogg, .opus, .m4a). Stereo calls are restored per channel. One channel holding both parties works too: v2 keeps the quieter voice.
• The default restores in 4-second windows with a 2-second crossfade. Keep it: the encoder's feature normalisation depends on clip length, and a single pass over a long file drops speech. Use --device cpu without a GPU.
• v1 stays at the tag v1. To run it, keep this script and fetch only v1's weights: hf download Scicom-intl/CallEnhancer fe_callcentre/fe_adapter_full.pt decoder_callcentre/decoder_only.pt --revision v1 --local-dir v1, then pass them to --fe-adapter and --decoder. v1's own script restores in a single pass by default.

Python

import numpy as np, soundfile as sf, torch, torchaudio
from huggingface_hub import hf_hub_download
from transformers import AutoFeatureExtractor
import infer_callcentre as ic          # from the download above, next to expand_decoder.py

torch.set_float32_matmul_precision("medium")                 # as the command line sets it: same output, bit for bit
dev = torch.device("cuda" if torch.cuda.is_available() else "cpu")
fe = ic.load_fe(hf_hub_download("Scicom-intl/CallEnhancer", "fe_callcentre/fe_adapter_full.pt"), dev)
dec, out_sr = ic.load_decoder(hf_hub_download("Scicom-intl/CallEnhancer", "decoder_callcentre/decoder_only.pt"), dev)
proc = AutoFeatureExtractor.from_pretrained("facebook/w2v-bert-2.0")

x, sr = sf.read("your_call.wav", dtype="float32", always_2d=True)
x16 = torchaudio.functional.resample(torch.from_numpy(x[:, 0])[None], sr, 16000)[0].numpy()   # one channel
x16 = x16 / (np.abs(x16).max() + 1e-9) * 0.95
y = ic.restore_channel(x16, fe, dec, proc, dev, chunk_s=4.0, bf16=True, out_sr=out_sr)
y = y / (np.abs(y).max() + 1e-9) * 0.97                       # peak level, as the command line writes it
sf.write("restored48k.wav", y, out_sr)

Files

path	what it is
fe_callcentre/fe_adapter_full.pt	encoder adapter for inference: 96 LoRA tensors and 48 output_dense biases on w2v-BERT 2.0
decoder_callcentre/decoder_only.pt	decoder for inference: the 188M DAC decoder, 48 kHz
decoder_callcentre/last.pt	the full training state at step 150,000 (decoder, adapter, discriminator, optimisers), to continue training
infer_callcentre.py, expand_decoder.py	the inference script and the decoder builder it imports
examples/	five public CallHome calls (TalkBank), each one channel with both parties: the input, and the v2 and v1 restorations
assets/	the figures on this card

Limitations

• The benchmarks are one operator's calls in English, Malay and Mandarin. Other lines and languages may differ.
• Sidon-style restoration resynthesises speech. Above 8 kHz the output is generated from the encoder's features, not recovered.
• SOTA-MOS mostly judges the band below 8 kHz. Use HBD and the spectra for the band above it.
• The test audio is customer data. We publish aggregates only, so the numbers cannot be reproduced outside Scicom.

License

CC BY-NC 4.0. Some training noise comes from non-commercial sources.

Built by Scicom (MSC) Berhad.