\pagestyle{fancy}
\cleardoublepage\chapter{Experiments in Chord Recognition}\label{chord_experiments}
\Quote{Celebrate any progress. Don't wait to get perfect.}{Ann McGee Cooper}

\noindent In this chapter, we will probe the various classes of image processing tools discussed in the preceding chapters in the context of the chord recognition problem. Remember that emphasis is laid here on horizontal structures in the spectrograms; accordingly, the algorithmic variants with horizontal directional preference will receive special attention.

Before entering the experiments, let us say a few words on the experimental setup. For all the computation, MATLAB 7.8.0 (R2009a) was executed on a 2.66 GHz Apple iMac with 4 GB of RAM running Mac OS X 10.6. The largest part of the code was implemented using the MATLAB programming language where in the case of CED\index{diffusion!coherence-enhancing} and EED\index{diffusion!edge-enhancing} the C code provided by the MIA group was used and slightly modified to compile as a MEX file that allows C code to run within the MATLAB environment \cite{mexref}.

The testfiles constitute excerpts of the songs referenced by their title and were annotated by Mauch et al.\ \cite{mauch09}. The name and the length of the excerpt of each testfile is listed in Table \ref{appendix.table_chord_files}. As a quality measure, precision\index{precision}, recall\index{recall} and F-measure\index{F-measure} as introduced in Section \ref{tasks.quality_measurement} are used.

%%% CHORD EXPERIMENTS: DIFFUSION %%%
\section{Diffusion\index{diffusion}}\label{chord_experiments.diffusion}

In this section, we show how diffusion\index{diffusion} algorithms perform on the spectrograms\index{spectrogram} for the task of chord recognition\index{chord recognition}.

In the first experiment, we look at the influence of using the diffusion tensor\index{diffusion!tensor} as introduced in Section \ref{modifications.diffusion} with and without the preference for horizontal structures for a fixed number of iterations. Secondly, we investigate the behavior of the algorithm for different numbers of iterations where $d_\alpha $ is set to 1 and $d_\beta$ is set to 0, i.e.\ we prefer a horizontal diffusion.

%%% CHORD EXPERIMENTS: EED %%%
\subsection{Edge-Enhancing Diffusion\index{diffusion!edge-enhancing}}

\paragraph*{Horizontal vs. no directional preference.} Figure \ref{experiments.chord.eed_preference} shows the spectrogram of ``Let It Be'' by The Beatles. The top spectrogram depicts the unprocessed data. In the bottom row, the comparison of an edge-enhancing diffusion\index{diffusion!edge-enhancing} without and with directional preference for horizontal structures is given. The bottom left spectrogram clearly shows that the diffusion happened in every direction which results also in a vertical blurring of the structures, i.e.\ the frequencies. In comparison to that, the version with directional preference turned on gives a spectrogram where the diffusion process only acts in horizontal direction which means that it does not blur other frequencies.
\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n10.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/eed/n10_100_1_1.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/eed/n10_100_0_1.eps}
	\caption[EED, comparison between directional preference and no preference.]{Spectrograms of ``Let It Be'' performed by The Beatles (number 10 in Table \ref{appendix.table_chord_files}). \textbf{Top:} Original spectrogram. \textbf{Bottom left:} Spectrogram after diffusion with EED and $\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$, $t = 100$ where no direction is preferred. \textbf{Bottom right:} Spectrogram after diffusion with the same setting, but with horizontal preference.}
	\label{experiments.chord.eed_preference}
\end{figure}

Table \ref{experiments.chord.eed_preference.table} also confirms the intuition that a diffusion process that diffuses in every direction does not help in the context of chord recognition\index{chord recognition}. In fact, the structures are blurred in such an intense way that the processed spectrogram is much less suited for feature extraction than the original, unprocessed spectrogram. Furthermore, the information which frequency contributes at which time is completely blurred away. This stands in contrast to the results obtained by the same computation but with preference for horizontal structures turned on. Here we can clearly distinguish the different frequencies involved in this music excerpt.

\begin{table}[ht]
	\centering
	\caption[EED, comparison between directional preference and no preference.]{Comparison of precision\index{precision}, recall\index{recall} and F-measure\index{F-measure} for EED with and without directional preference. Edge-enhancing diffusion was performed with $\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$ and $t = 100$. The values correspond to the processing depicted in Figure \ref{experiments.chord.eed_preference}.}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Setting & Precision & Recall & F-Measure \\
		\hline
		Original & 0.496 & 0.496 & 0.496 \\
		No preference & 0.021 & 0.021 & 0.021 \\
		Horizontal preference & 0.585 & 0.585 & 0.585
	\end{tabular}
	\label{experiments.chord.eed_preference.table}
\end{table}

\paragraph*{Increasing the number of iterations.} In addition to that, Figure \ref{experiments.chord.eed_iterations} shows spectrograms processed by edge-enhancing diffusion\index{diffusion!edge-enhancing} with directional preference for horizontal structures where the impact of different numbers of iterations is compared. Obviously, the more iterations are performed the stronger the smearing in time. In other words, long diffusion times introduce an error since frequencies appear where they are actually not located. Table \ref{experiments.chord.eed_iterations.table} shows that the higher the number of iterations the lower the F-measure of the result. Let us take as an example the peak located at a frequency slightly below $10^2$ that extends in time from shortly before 5 to about 6 seconds. We see that this frequency contributes more and more beyond the original boundaries of this time frame the higher $t$ gets. For $t = 500$ we can observe that a contribution is present almost at every time in this example.

Please also note that for this particular testfile the values for precision\index{precision} and recall\index{recall} given in tables \ref{experiments.chord.eed_preference.table} and \ref{experiments.chord.eed_iterations.table} are the same -- also resulting in the same value for the F-measure\index{F-measure}, compare equation \eqref{eq_fmeasure} -- which does not hold in general.
\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n10.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/eed/n10_50_0_1.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/eed/n10_100_0_1.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/eed/n10_500_0_1.eps}
	\caption[EED, comparison between different numbers of iterations.]{Spectrograms of ``Let It Be'' performed by The Beatles (number 10 in Table \ref{appendix.table_chord_files}). All diffusion processes are performed with a preference for horizontal structures. \textbf{Top left:} Original spectrogram. \textbf{Top right:} Spectrogram after diffusion with EED and $\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$ and $t = 50$. \textbf{Bottom left:} $t = 100$. \textbf{Bottom right:} $t = 500$.}
	\label{experiments.chord.eed_iterations}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[EED, comparison between different numbers of iterations.]{Comparison of precision\index{precision}, recall\index{recall} and F-measure\index{F-measure} for EED with different numbers of iterations and directional preference for horizontal structures where $\lambda = 1$, $\sigma = 0.5$ and $\tau = 0.25$. The values correspond to the processing depicted in Figure \ref{experiments.chord.eed_iterations}.}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Iterations & Precision & Recall & F-Measure \\
		\hline
		0 & 0.496 & 0.496 & 0.496 \\
		50 & 0.634 & 0.634 & 0.634 \\
		100 & 0.585 & 0.585 & 0.585 \\
		500 & 0.475 & 0.475 & 0.475
	\end{tabular}
	\label{experiments.chord.eed_iterations.table}
\end{table}

%%% CHORD EXPERIMENTS: CED %%%
\subsection{Coherence-Enhancing Diffusion\index{diffusion!coherence-enhancing}}

\paragraph*{Horizontal vs. no directional preference.} Turning to coherence-enhancing diffusion, we start by investigating the impact of steering the diffusion only in horizontal direction in comparison to a diffusion process that diffuses in every direction like we did in the previous section. Figure \ref{experiments.chord.ced_preference} depicts the corresponding spectrograms for ``All You Need Is Love'' by The Beatles. On the bottom left spectrogram, we can see that unbiased anisotropic diffusion heavily blurs the structures in the spectrogram. It seems that in the top half of the spectrogram every frequency contributes at every time. This result is far away from being useful which is also affirmed by the quality measure, compare Table \ref{experiments.chord.ced_preference.table}. Again, the change of the diffusion tensor to only allow horizontal diffusion helps to get rid off this instance (Figure \ref{experiments.chord.ced_preference}, bottom right). Despite the presence of blurring in the top half of the spectrogram, we may see that the frequencies in the bottom of the spectrogram are well preserved by this modified diffusion. % In contrast, on the bottom right spectrogram of Figure \ref{experiments.chord.ced_preference} the horizontal preference of the diffusion process is clearly visible.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n2.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/ced/n2_500_1_1.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/ced/n2_500_0_1.eps}
	\caption[CED, comparison between directional preference and no preference.]{Spectrograms of ``All You Need Is Love'' performed by The Beatles (number 2 in Table \ref{appendix.table_chord_files}). \textbf{Top:} Original spectrogram. \textbf{Bottom left:} Spectrogram after diffusion with CED and $C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 0.1$, $\tau = 0.25$, $t = 500$ where no direction is preferred. \textbf{Bottom right:} Spectrogram after diffusion with the same setting, but with preference for horizontal direction.}
	\label{experiments.chord.ced_preference}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[CED, comparison between directional preference and no preference.]{Comparison of precision\index{precision}, recall\index{recall} and F-measure\index{F-measure} for CED with and without directional preference where $C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 0.1$, $\tau = 0.25$ and $t = 500$. The values correspond to the processing depicted in Figure \ref{experiments.chord.ced_preference}.}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Setting & Precision & Recall & F-Measure \\
		\hline
		Original & 0.412 & 0.416 & 0.414 \\
		No preference & 0.000 & 0.000 & 0.000 \\
		Horizontal preference & 0.775 & 0.783 & 0.779
	\end{tabular}
	\label{experiments.chord.ced_preference.table}
\end{table}

\paragraph*{Increasing the number of iterations.} Analyzing Figure \ref{experiments.chord.ced_iterations} and the corresponding Table \ref{experiments.chord.ced_iterations.table} we notice something counterintuitive. The more the bright, yellowish frequencies in the bottom half of the spectrogram are blurred, the more enhanced is the result according to the quality measure. We would expect a poor outcome for high iteration numbers, i.e.\ an higher error with the increase of the number of iterations and thus smaller values for precision\index{precision}, recall\index{recall} and F-measure\index{F-measure}, but receive the opposite. The reason is the following. Diffusion in horizontal direction introduces the frequencies in the bottom of the spectrogram more and more at every point in time the more iterations are performed. This results in the existence of strong contributions of these frequencies where actual occurrences should be located while the contributions that are spuriously introduced are small securing the result a high quality rating according to the precision\index{precision}, recall\index{recall} and F-measure\index{F-measure} criteria. This demonstrates that these criteria are not always suitable as measures of quality, and their output should be taken with some caution.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n2.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/ced/n2_100_0_1.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/ced/n2_500_0_1.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/ced/n2_1000_0_1.eps}
	\caption[CED, comparison between different numbers of iterations.]{Spectrograms of ``All You Need Is Love'' performed by The Beatles (number 2 in Table \ref{appendix.table_chord_files}). All diffusion processes are performed with a preference for horizontal structures. \textbf{Top left:} Original spectrogram. \textbf{Top right:} Spectrogram after diffusion with CED and $C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 0.1$, $\tau = 0.25$, $t = 100$. \textbf{Bottom left:} $t = 500$. \textbf{Bottom right:} $t = 1000$.}
	\label{experiments.chord.ced_iterations}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[CED, comparison between different numbers of iterations.]{Comparison of precision\index{precision}, recall\index{recall} and F-measure\index{F-measure} for CED ($C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 0.1$ and $\tau = 0.25$) with horizontal preference and different numbers of iterations. The values correspond to the processing depicted in Figure \ref{experiments.chord.ced_iterations}.}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Iterations & Precision & Recall & F-Measure \\
		\hline
		0 & 0.412 & 0.416 & 0.414 \\
		100 & 0.569 & 0.575 & 0.572 \\
		500 & 0.775 & 0.783 & 0.779 \\
		1000 & 0.902 & 0.910 & 0.906
	\end{tabular}
	\label{experiments.chord.ced_iterations.table}
\end{table}

%%% CHORD EXPERIMENTS: EED vs. CED %%%
\clearpage
\subsection{Comparison of EED and CED}\label{chord_experiments.eed_vs_ced}
We have seen that EED and CED can perform similarly well in the setting of chord recognition\index{chord recognition}. We conclude this section by comparing edge-enhancing diffusion\index{diffusion!edge-enhancing} and coherence-enhancing diffusion\index{diffusion!coherence-enhancing} head to head with similar parameter settings. Please note that the parameter $\alpha$ in CED is some sort of damping parameter for the diffusion process. If we scale $\alpha$ down by a factor $s$, we have to perform $t \cdot s$ iterations to get the same result. For the following comparison, we set $\alpha = 1$ to ensure that both EED and CED run with the same number of iterations.

Visually, no difference between the spectrograms on the left and on the right of Figure \ref{experiments.chord.eed_ced_comparison} can be identified. Also the quality measurements given in Table \ref{experiments.chord.eed_ced_comparison.table} do not disclose any differences. The natural question is why is this the case? The answer is simple. Coherence-enhancing diffusion enhances flow-like (coherent) structures. In this context, these flow-like structures have the same orientation as the edges. Thus, edge-enhancing diffusion\index{diffusion!edge-enhancing} and coherence-enhancing diffusion\index{diffusion!coherence-enhancing} compute a similar structure tensor\index{structure tensor} which results in a similar diffusion behavior. %a similar diffusion tensor\index{diffusion!tensor}.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n8.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/eed/n8_100_1_1.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/ced/n8_100_1_1.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/eed/n8_100_0_1.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/ced/n8_100_0_1.eps}
	\caption[Comparison between EED and CED.]{Comparison between EED and CED. \textbf{Top:} Original spectrogram of ``I Am The Walrus'' by The Beatles (number 8 in Table \ref{appendix.table_chord_files}). \textbf{Middle left:} Edge-enhancing diffusion with $\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$, $t = 100$ and no preference for horizontal diffusion. \textbf{Bottom left:} Same setting with preference for horizontal diffusion. \textbf{Middle right:} Coherence-enhancing diffusion with $C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 1$, $\tau = 0.25$, $t = 100$ and no preference for horizontal diffusion. \textbf{Bottom right:} Same parameters, but directional preference for horizontal diffusion turned on.}
	\label{experiments.chord.eed_ced_comparison}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[Comparison between EED and CED.]{Comparison between EED with $\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$, $t = 100$ and CED using $C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 1$, $\tau = 0.25$, $t = 100$. The table corresponds to Figure \ref{experiments.chord.eed_ced_comparison}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		Method & Preference & Precision & Recall & F-Measure \\
		\hline
		Original & -- & 0.456 & 0.464 & 0.460 \\
		\hline
		EED & no & 0.000 & 0.000 & 0.000 \\
		CED & no & 0.000 & 0.000 & 0.000 \\
		\hline
		EED & yes & 0.562 & 0.573 & 0.567 \\
		CED & yes & 0.562 & 0.573 & 0.567
	\end{tabular}
	\label{experiments.chord.eed_ced_comparison.table}
\end{table}

%%% CHORD EXPERIMENTS: MORPHOLOGY %%%
\clearpage
\section{Morphology\index{morphology}}\label{chord_experiments.morphology}

This section deals with morphological operations. After experiments that show how the basic building blocks of morphology, i.e.\ dilation and erosion as well as opening and closing, work on spectrograms\index{spectrogram} we proceed by focussing on skeletonization\index{skeletonization} where the testfile ``Baby's In Black'' by The Beatles (depicted in Figure \ref{experiments.chord.morphology.original}) is used.

\begin{figure}[ht]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n4.eps}
	\caption[Spectrogram of ``Baby's In Black'' by The Beatles.]{Spectrogram of ``Baby's In Black'' by The Beatles (number 4 in Table \ref{appendix.table_chord_files}).}
	\label{experiments.chord.morphology.original}
\end{figure}

%%% CHORD EXPERIMENTS: DILATION/EROSION %%%
\subsection{Dilation\index{dilation} and Erosion\index{erosion}}

First, we start by investigating the building blocks dilation\index{dilation} and erosion\index{erosion} using a rectangle and a disk as structuring element. The top row of Figure \ref{experiments.chord.morphology.rect} depicts the outcome of dilation and erosion using a rectangle of height 1 and width 10 as structuring element where the top row of Figure \ref{experiments.chord.morphology.disk} shows the results for the same computation but with a disk of radius 10 as structuring element. It is clearly visible that dilation blurs the original spectrogram a lot for both structuring elements. While the top left spectrogram in Figure \ref{experiments.chord.morphology.rect} allows to guess the original arrangement of frequencies, this information is completely destroyed in the top left spectrogram of Figure \ref{experiments.chord.morphology.disk}. The erosion which is depicted for a rectangle and a disk as structuring element in the top right of Figure \ref{experiments.chord.morphology.rect} and Figure \ref{experiments.chord.morphology.disk}, respectively, we find out that this operation throws a lot of information away. Similarly to dilation with a disk, erosion with a disk structuring element degenerates the final result in such a way that it is practically not useful. In contrast, erosion with a rectangle removes almost every frequency contribution while preserving the basic structure such that the most important frequencies for chord recognition are still available, compare Table \ref{experiments.chord.morphology.rect.table}.% Interestingly, according to the quality measure, the result for erosion is good enough, compare Table \ref{experiments.chord.morphology.rect.table}. This is due to the fact that all frequency contributions except for strong ones are eroded away resulting in the fact that the most important frequencies for chord recognition are still available.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_dil_rect_1_10.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_ero_rect_1_10.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_close_rect_1_10.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_open_rect_1_10.eps}
	\caption[Morphological operations on the spectrogram of ``Baby's In Black''.]{Morphological operations on the spectrogram of ``Baby's In Black'' by The Beatles with a rectangle of height 1 and width 10 as structuring element. \textbf{Top left:} Dilation. \textbf{Top right:} Erosion. \textbf{Bottom left:} Closing. \textbf{Bottom right:} Opening.}
	\label{experiments.chord.morphology.rect}
\end{figure}

\begin{table}[hb]
	\centering
	\caption[Morphological operations on the spectrogram of ``Baby's In Black''.]{Morphological operations on the spectrogram of ``Baby's In Black'' by The Beatles with a rectangle of height 1 and width 10 as structuring element. See also Figure \ref{experiments.chord.morphology.rect}.}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.475 & 0.482 & 0.479 \\
		\hline
		Dilation & 0.438 & 0.445 & 0.442 \\
		Erosion & 0.714 & 0.725 & 0.720 \\
		\hline
		Closing & 0.505 & 0.512 & 0.509 \\
		Opening & 0.722 & 0.733 & 0.727
	\end{tabular}
	\label{experiments.chord.morphology.rect.table}
\end{table}

Generally speaking, using a disk as structuring element is not an optimal choice since a disk destroys -- by virtue of its shape -- the frequency distribution in the spectrogram. This fact is also supported by the quality measure, see Table \ref{experiments.chord.morphology.disk.table}.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_dil_disk_10_0.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_ero_disk_10_0.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_close_disk_10_0.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_open_disk_10_0.eps}
	\caption[Morphological operations on the spectrogram of ``Baby's In Black''.]{Morphological operations on the spectrogram of ``Baby's In Black'' by The Beatles with a disk of radius 10 as structuring element. \textbf{Top left:} Dilation. \textbf{Top right:} Erosion. \textbf{Bottom left:} Closing. \textbf{Bottom right:} Opening.}
	\label{experiments.chord.morphology.disk}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[Morphological operations on the spectrogram of ``Baby's In Black''.]{Morphological operations on the spectrogram of ``Baby's In Black'' by The Beatles with a disk of radius 10 as structuring element. See also Figure \ref{experiments.chord.morphology.disk}.}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.475 & 0.482 & 0.479 \\
		\hline
		Dilation & 0.039 & 0.040 & 0.040 \\
		Erosion & 0.017 & 0.018 & 0.017 \\
		\hline
		Closing & 0.204 & 0.207 & 0.206 \\
		Opening & 0.067 & 0.068 & 0.067
	\end{tabular}
	\label{experiments.chord.morphology.disk.table}
\end{table}

%These investigations show that dilation and erosion using a disk as structuring element are not very useful. We will now investigate opening and closing with which we can try to enhance the results given by dilation or erosion.

%%% CHORD EXPERIMENTS: OPENING/CLOSING %%%
\clearpage
\subsection{Opening\index{opening} and Closing\index{closing}}

Figure \ref{experiments.chord.morphology.rect} shows on the bottom the spectrograms resulting from applying closing and opening to the spectrogram given in Figure \ref{experiments.chord.morphology.original}. Let us recall that a closing operation is a dilation followed by an erosion and an opening operation is the opposite (first erosion, then dilation). Please note that viewing the spectrograms in Figure \ref{experiments.chord.morphology.rect} column-wise gives in the top row automatically the intermediate step for the operation depicted in the bottom row.

We investigate the closing in Figure \ref{experiments.chord.morphology.rect} first. The blurring introduced by dilation is slightly removed by the followed erosion resulting in an improvement of the outcome, compare Table \ref{experiments.chord.morphology.rect.table}. We have seen before that applying erosion performs well for the chord recognition process since it removes mainly noisy frequency contributions. Applying a dilation to the erosion step brings some contributions back. This can be seen in the bottom right of Figure \ref{experiments.chord.morphology.rect} where the opening on the spectrogram given in Figure \ref{experiments.chord.morphology.original} is shown. The improvement of the opening over simply performing erosion is also supported by the values precision, recall and F-measure given in Table \ref{experiments.chord.morphology.rect.table}.

As done in the section before, we briefly touch the result computed with the disk structuring element as given in Figure \ref{experiments.chord.morphology.disk}. In this case we see that closing and opening cannot improve the result very much. In fact, the results in Table \ref{experiments.chord.morphology.disk.table} show that opening and closing bring enhancements to the results computed by an erosion or dilation, but still do not improve the final result compared to the original. This circumstance is due to the situation already mentioned before, i.e.\ that the disk shape destroys too much information about the frequency distribution.

%%% CHORD EXPERIMENTS: SKELETONIZATION %%%
\subsection{Skeletonization\index{skeletonization}}\label{chord_experiments.skeletonization}

%We have seen in Section \ref{morphology.skeletonization} that a skeletonization algorithm can be realized by the use of dilation and erosion.
In the following, we focus on the \emph{parabolic skeletonization} algorithm introduced in Section \ref{modifications.morphology} and we refer to it by simply using \emph{skeletonization}. Thus, let us first show the application of parabolic dilation and parabolic erosion on ``Baby's In Black'' by The Beatles. The processed spectrograms are given in Figure \ref{experiments.chord.pardil_parero}. We immediately notice that neither the application of parabolic dilation nor the application of parabolic erosion results in such extreme changes of the structure in the spectrograms as investigated in Figures \ref{experiments.chord.morphology.rect} and \ref{experiments.chord.morphology.disk}. Both, parabolic dilation and parabolic erosion, blur the frequencies marginally where a small blow-up can be noticed for parabolic dilation and a small shrinkage for parabolic erosion while basically preserving the overall structure of the spectrogram.

\begin{figure}[tb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_pardil_01_01.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_parero_01_01.eps}
	\caption[Parabolic morphological operations on the spectrogram of ``Baby's In Black''.]{Parabolic morphological operations on the spectrogram of ``Baby's In Black'' by The Beatles. \textbf{Left:} Parabolic dilation with $s_x = 0.1$ and $s_y = 0.1$. \textbf{Right:} Parabolic erosion with $s_x = 0.1$ and $s_y = 0.1$.}
	\label{experiments.chord.pardil_parero}
\end{figure}

MATLAB comes with a built-in skeletonization function (\texttt{bwmorph}) that only works on binary images. The problem, though, with this approach is to find a suitable threshold to compute a binary version of the grayscale image. MATLAB also provides a function for this task (\texttt{graythresh}) which computes a global image threshold following the method proposed by N. Otsu \cite{otsu79}. The binary image can be computed using \texttt{im2bw} (also a MATLAB function). Furthermore, the result can be slightly enhanced by using the color information of the original spectrogram at the skeleton points. Figure \ref{experiments.chord.bwmorph.skel} shows the binary image of the ``Baby's In Black'' spectrogram as well as its skeleton computed by \texttt{bwmorph}. The result of this procedure is a skeleton with thin, sharp edges which is usually disadvantageous. Music is dependent on things like overtones that are removed by the application of \texttt{graythresh}. Due to this fact, the result is -- though it might be of good visual quality -- not well suited for chord recognition which can also be seen in Table \ref{experiments.chord.skeletonization.table}.

\begin{figure}[tb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n4.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n4_bw.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_bwskel.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_bwskel_color.eps}
	\caption[Skeletonization algorithm as implemented in MATALB.]{Skeletonization of ``Baby's In Black'' by The Beatles. \textbf{Top left:} Original spectrogram. \textbf{Top right:} Binary image of the original spectrogram generated using the built-in MATLAB function \texttt{im2bw} with an automatically chosen threshold realized by the built-in function \texttt{graythresh}. \textbf{Bottom left:} Skeleton computed with the MATLAB function \texttt{bwmorph} using the binary version of the original spectrogram. \textbf{Bottom right:} Same skeleton with color information of the original spectrogram at skeleton points.}
	\label{experiments.chord.bwmorph.skel}
\end{figure}

\begin{figure}[tb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_bwskel_color.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_skel_01_01_0001.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_skel_1_01_0001.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/morphology/n4_skel_01_1_0001.eps}
	\caption[Different skeletonization approaches on ``Baby's In Black''.]{Different skeletonization approaches on ``Baby's In Black'' by The Beatles. \textbf{Top left:} Skeleton computed with \texttt{bwmorph} and additional color information. \textbf{Top right:} Parabolic skeletonization with $s_x = 0.1$, $s_y = 0.1$ and a threshold of $0.001$. \textbf{Bottom left:} Same, but $s_x = 1$, $s_y = 0.1$. \textbf{Bottom right:} Same, but $s_x = 0.1$, $s_y = 1$.}
	\label{experiments.chord.skeletonization}
\end{figure}

\begin{table}[b]
	\centering
	\caption[Comparison of skeletonization algorithms on ``Baby's In Black''.]{Comparison of skeletonization algorithms on ``Baby's In Black'' by The Beatles. The first two skeletonizations correspond to the \texttt{bwmorph} function that is built into MATLAB. The binary image for \texttt{bwmorph} was generated using \texttt{im2bw} with an automatically chosen threshold realized by \texttt{graythresh}. Last three are parabolic skeletonizations with a threshold of $0.001$ for different values for $s_x$ and $s_y$.}
	\ \\[0.3cm]
	\begin{tabular}{cccccc}
		Method & $s_x$ & $s_y$ & Precision & Recall & F-measure \\
		\hline
		Original & -- & -- & 0.475 & 0.482 & 0.479 \\
		\hline
		MATLAB & -- & -- & 0.419 & 0.425 & 0.422 \\
		MATLAB + color & -- & -- & 0.433 & 0.440 & 0.437 \\
		\hline
		Parabolic & 0.1 & 0.1 & 0.537 & 0.545 & 0.541 \\
		Parabolic & 1 & 0.1 & 0.515 & 0.522 & 0.519 \\
		Parabolic & 0.1 & 1 & 0.433 & 0.440 & 0.437
	\end{tabular}
	\label{experiments.chord.skeletonization.table}
\end{table}

Figure \ref{experiments.chord.skeletonization} compares the skeleton computed with \texttt{bwmorph} plus additional color information with the skeletons resulting from the application of the parabolic skeletonization algorithm with different values for $s_x$ and $s_y$. The difference between the \texttt{bwmorph} skeleton and the parabolic ones is quite obvious: the skeletons computed with parabolic skeletonization introduce some blurring while the \texttt{bwmorph} skeleton has thin, sharp edges. As already mentioned, the parabolic skeletons preserve information necessary for music information retrieval. Comparing these skeletons among each other, hardly any difference can be seen. The skeleton produced with $s_x = 1$ and $s_y = 0.1$ shows a little preference for horizontal structures where the result using $s_x = 0.1$ and $s_y = 1$ gives more a vertical preference.

Due to the quality measure, the parabolic skeletonization with $s_x = 0.1$ and $s_y = 0.1$ gives the best result for this testfile, compare Table \ref{experiments.chord.skeletonization.table}. However, the outcome depends on the input data. For some files, an improved result for $s_x = 1$ and $s_y = 0.1$ can be seen, but in general, $s_x = s_y = 0.1$ is a reasonable starting point.

%%% CHORD EXPERIMENTS: NL-MEANS %%%
\clearpage
\section{NL-Means\index{NL-means}}\label{chord_experiments.nlmeans}

In Sections \ref{nlmeans.refinements} and \ref{modifications.nlmeans}, we have seen that various improvements can be made to NL-means\index{NL-means} to make the algorithm faster. These improvements also introduce assumptions and modifications to the originally intended algorithm. We will investigate in this section how these assumptions and changes influence the outcome of the NL-means\index{NL-means} algorithm for chord recognition. In our experiments, we omit the usage of the Gaussian convolution of the Euclidean distance as shown in equation \eqref{nlmeans_equation}. The influence of this modification on the result is negligible because the Gaussian convolution only slightly modifies the NL-means\index{NL-means} filter. In addition, the runtime of the algorithm is a bit lower. Due to the computational complexity, we do not consider the basic version where the search space for similar patches is not restricted.

In the following -- and also in Chapter \ref{onset_experiments} -- we stick to the term \emph{NL-means} if we do not consider the modifications presented in Section \ref{nlmeans.neighborhood_classification} and to \emph{fast NL-means} if the modified weighting function as given in equation \eqref{nlmeans_fast_weighting_function} is used. For both approaches, NL-means and fast NL-means, we consider all other improvements such as the locally adaptive weighting and the modified search and similarity windows.

\subsection{Improved NL-means}
We start with the comparison between NL-means\index{NL-means} with and without adaptive weighting (see Section \ref{nlmeans.adaptive_weighting}) for different parameters $\lambda$.
\paragraph*{No directional preference.} In this experiment, no directional preference -- i.e.\ a quadratic similarity window and a quadratic search window -- is used. Figure \ref{experiments.chord.nodirection.nonadaptive_adaptive} shows the spectrograms of ``Tell Me Why'' by The Beatles. In the left column of the figure, we see the resulting spectrograms for different values of $\lambda$ without adaptive weighting. In the right column, exactly the same computation was performed but with adaptively changing the values of the distance weight $\lambda$ according to equation \eqref{nlmeans_lambda_adaptive_weighting}. It is clearly visible that the algorithm blurs structures more if $\lambda$ is not chosen adaptively. This coincides with the fact that a globally chosen $\lambda$ is not always preferable, see also \cite{brox07, zimmer08}. The quality measure given in Table \ref{experiments.chord.nodirection.nonadaptive_adaptive.table} shows that for $\lambda = 10$ this does not hold for the examined testfile, but in general, an adaptively chosen distance weight $\lambda$ gives either the same result or improves it in comparison to the version without adaptation.

The best result in Figure \ref{experiments.chord.nodirection.nonadaptive_adaptive} is achieved for $\lambda = 1$ in conjunction with adaptive weighting which can also be visually verified. Fine structures like small contributions in the top quarter of the spectrogram are enhanced pretty well compared to the version without adaptive weighting where detailed structures are unsharp after the application of the algorithm due to the heavy blurring involved. Moreover, points where no or small contributions can be found -- i.e.\ the dark parts in the spectrograms -- are filled up with contributions the higher $\lambda$ gets.% This is due to the weighted averaging computed for each pixel, since it consists of contributions lying in the similarity window and the higher $\lambda$ is chosen, the higher the weighting and thus the contribution itself becomes.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n12.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeans/n12_10_10_3_3_1_n.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeans/n12_10_10_3_3_1_y.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeans/n12_10_10_3_3_5_n.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeans/n12_10_10_3_3_5_y.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeans/n12_10_10_3_3_10_n.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeans/n12_10_10_3_3_10_y.eps}
	\caption[NL-means, comparison of choosing $\lambda$ adaptively or not, quadratic windows.]{Comparison of choosing $\lambda$ adaptively or not for NL-means using the testfile ``Tell Me Why'' by The Beatles (number 12 in Table \ref{appendix.table_chord_files}). All spectrograms were computed using a $21 \times 21$ search window and a $7 \times 7$ similarity window. \textbf{Top:} Original spectrogram. \textbf{Left:} $\lambda$ is not chosen adaptively. \textbf{Right:} $\lambda$ is chosen adaptively. \textbf{Middle to bottom:} $\lambda = 1, 5, 10$.}
	\label{experiments.chord.nodirection.nonadaptive_adaptive}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[NL-means, comparison of choosing $\lambda$ adaptively or not, quadratic windows.]{Comparison of choosing $\lambda$ adaptively or not for NL-means with a $21 \times 21$ search window and a $7 \times 7$ similarity window. This table corresponds to Figure \ref{experiments.chord.nodirection.nonadaptive_adaptive}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		$\lambda$ & Adaptive & Precision & Recall & F-measure \\
		\hline
		Original & -- & 0.213 & 0.221 & 0.217 \\
		\hline
		1 & no & 0.142 & 0.147 & 0.145 \\
		1 & yes & 0.256 & 0.266 & 0.261 \\
		\hline
		5 & no & 0.062 & 0.064 & 0.063 \\
		5 & yes & 0.071 & 0.074 & 0.072 \\
		\hline
		10 & no & 0.059 & 0.061 & 0.060 \\
		10 & yes & 0.049 & 0.051 & 0.050
	\end{tabular}
	\label{experiments.chord.nodirection.nonadaptive_adaptive.table}
\end{table}

\paragraph*{Horizontal directional preference.} In addition to the previous experiment, we have a look at how a directional preference -- i.e.\ a rectangular similarity window (compare Section \ref{modifications.nlmeans}) -- changes the result. We stick to the same experimental setting as in the experiment before, but change the height of the similarity window to 3, i.e.\ we use a similarity window of size $7 \times 3$. The spectrograms belonging to this experiment are given in Figure \ref{experiments.chord.direction.nonadaptive_adaptive}. Regarding this figure, the first thing we notice is that horizontal structures dominate. Let us compare Tables \ref{experiments.chord.nodirection.nonadaptive_adaptive.table} and \ref{experiments.chord.direction.nonadaptive_adaptive.table}. First, improvements are achieved when using a rectangular window in conjunction with the usage of adaptive weighting. Second, for the setting where adaptive weighting is not used, no differences between a $7 \times 7$ and a $7 \times 3$ similarity window can be found. An exception for both cases is $\lambda = 1$.

The main reason to only restrict the similarity window -- i.e.\ the inpainting window -- is to make the search for similar structures in the spectrogram more reliable. Let us assume for the moment a $21 \times 3$ search window and a $7 \times 3$ similarity window. We shortly give here the corresponding values for precision\index{precision}, recall\index{recall} and F-measure\index{F-measure} for $\lambda = 1$. For the variant where $\lambda$ is not computed adaptively we have $(P, R, F) = (0.130, 0.135, 0.132)$ and for the variant with adaptive weighting, we have $(P, R, F) = (0.198, 0.205, 0.201)$. Comparing these values with the corresponding ones in Table \ref{experiments.chord.direction.nonadaptive_adaptive.table}, we notice higher values for precision\index{precision}, recall\index{recall} and F-measure\index{F-measure}, meaning that the result computed with a $21 \times 21$ search window is advantageous.
%Surprisingly, for $\lambda = 5$ we get $(P, R, F) = (0.136, 0.141, 0.138)$ for the nonadaptive version and $(P, R, F) = (0.148, 0.154, 0.151)$ for the adaptive version and for $\lambda = 10$ we get $(P, R, F) = (0.139, 0.144, 0.142)$ and $(P, R, F) = (0.154, 0.160, 0.157)$ for the nonadaptive version and the adaptive version, respectively.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n12.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeans/n12_10_10_3_1_1_n.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeans/n12_10_10_3_1_1_y.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeans/n12_10_10_3_1_5_n.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeans/n12_10_10_3_1_5_y.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeans/n12_10_10_3_1_10_n.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeans/n12_10_10_3_1_10_y.eps}
	\caption[NL-means, comparison of choosing $\lambda$ adaptively or not, rectangular window.]{Same as in Figure \ref{experiments.chord.nodirection.nonadaptive_adaptive}, but with a $21 \times 21$ search window and a $7 \times 3$ similarity window.}
	\label{experiments.chord.direction.nonadaptive_adaptive}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[NL-means, comparison of choosing $\lambda$ adaptively or not, rectangular window.]{Comparison of choosing $\lambda$ adaptively or not for NL-means with a $21 \times 21$ search window and a $7 \times 3$ similarity window. This table corresponds to Figure \ref{experiments.chord.direction.nonadaptive_adaptive}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		$\lambda$ & Adaptive & Precision & Recall & F-measure \\
		\hline
		Original & -- & 0.213 & 0.221 & 0.217 \\
		\hline
		1 & no & 0.151 & 0.157 & 0.154 \\
		1 & yes & 0.216 & 0.224 & 0.220 \\
		\hline
		5 & no & 0.062 & 0.064 & 0.063 \\
		5 & yes & 0.114 & 0.119 & 0.116 \\
		\hline
		10 & no & 0.059 & 0.061 & 0.060 \\
		10 & yes & 0.068 & 0.071 & 0.069
	\end{tabular}
	\label{experiments.chord.direction.nonadaptive_adaptive.table}
\end{table}

\clearpage
\subsection{Fast NL-means}
In this paragraph, let us discuss the fast NL-means algorithm (compare Section \ref{nlmeans.neighborhood_classification}) with and without adaptive weighting for different $\lambda$.
\paragraph*{No directional preference.} We start again with a quadratic window for the search as well as for the similarity window. Furthermore, we investigate how the two parameters $\eta_1$ and $\eta_2$ change the result. These parameters are a crucial threshold in the fast NL-means algorithm which can be seen in equation \eqref{nlmeans_fast_weighting_function}.

In Figure \ref{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset1}, the processing of ``Tell Me Why'' (number 12 in Table \ref{appendix.table_chord_files}) with fast NL-means is shown. To compute these spectrograms, $\eta_1 = 0.9$ and $\eta_2 = 1.1$ are chosen. Except for the computation with $\lambda = 1$ and adaptive weighting -- which enhances the result compared to the original -- all the other parameter settings did not improve the spectrogram at all, see Table \ref{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset1.table}. It is also clearly visible that in all the spectrograms that did not enhance the original a lot of blurring is present that does not preserve the fine detailed structures.% In this sense, the fast NL-means\index{NL-means} algorithm behaves like the NL-means\index{NL-means} algorithm examined before.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n12.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_3_1_n_09_11.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_3_1_y_09_11.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_3_5_n_09_11.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_3_5_y_09_11.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_3_10_n_09_11.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_3_10_y_09_11.eps}
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, quadratic windows, $\eta_1 = 0.9$, $\eta_2 = 1.1$.]{Comparison of choosing $\lambda$ adaptively or not for fast NL-means using the testfile ``Tell Me Why'' by The Beatles (number 12 in Table \ref{appendix.table_chord_files}). All spectrograms were computed using a $21 \times 21$ search window, a $7 \times 7$ similarity window, $\eta_1 = 0.9$ and $\eta_2 = 1.1$. \textbf{Top:} Original spectrogram. \textbf{Left:} $\lambda$ is not chosen adaptively. \textbf{Right:} $\lambda$ is chosen adaptively. \textbf{Middle to bottom:} $\lambda = 1, 5, 10$.}
	\label{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset1}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, quadratic windows, $\eta_1 = 0.9$, $\eta_2 = 1.1$.]{Comparison of choosing $\lambda$ adaptively or not for fast NL-means with a $21 \times 21$ search window, a $7 \times 7$ similarity window, $\eta_1 = 0.9$ and $\eta_2 = 1.1$. This table corresponds to Figure \ref{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset1}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		$\lambda$ & Adaptive & Precision & Recall & F-measure \\
		\hline
		Original & -- & 0.213 & 0.221 & 0.217 \\
		\hline
		1 & no & 0.179 & 0.186 & 0.182 \\
		1 & yes & 0.253 & 0.263 & 0.258 \\
		\hline
		5 & no & 0.142 & 0.147 & 0.145 \\
		5 & yes & 0.160 & 0.167 & 0.164 \\
		\hline
		10 & no & 0.142 & 0.147 & 0.145 \\
		10 & yes & 0.142 & 0.147 & 0.145
	\end{tabular}
	\label{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset1.table}
\end{table}

Figure \ref{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset2} shows the same experiments by using a $21 \times 21$ search window and a $7 \times 7$ similarity window where $\eta_1 = 0.5$ and $\eta_2 = 1.5$ are used. Intuitively, we know that similar neighborhoods should have similar gray values. Increasing the range given in equation \eqref{nlmeans_avg_grayvalue_frac} means that we are more tolerant when it comes to deviations from the average gray value in the two windows that are compared. Let us investigate the spectrograms in Figure \ref{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset1} and the corresponding spectrograms in Figure \ref{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset2}. We observe that for $\eta_1 = 0.5$ and $\eta_2 = 1.5$ the spectrograms are much more dense (smeared) in comparison to the ones computed with $\eta_1 = 0.9$ and $\eta_2 = 1.1$. This visual observation is also supported by the measurement of precision\index{precision}, recall\index{recall} and F-measure\index{F-measure} given in Table \ref{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset1.table} and \ref{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset2.table} where the result has in general a higher F-measure for $\eta_1 = 0.9$ and $\eta_2 = 1.1$ except for $\lambda = 1$ with adaptive weighting where the version with $\eta_1 = 0.5$ and $\eta_2 = 1.5$ gives a slight improvement. Note that choosing $\eta_1$ and $\eta_2$ further away from 1 -- as in the setting with $\eta_1 = 0.5$ and $\eta_2 = 1.5$ -- may increase the quality of the denoised version but can also introduce a higher error since more contributions are considered to compute the averaged weighting.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n12.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_3_1_n_05_15.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_3_1_y_05_15.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_3_5_n_05_15.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_3_5_y_05_15.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_3_10_n_05_15.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_3_10_y_05_15.eps}
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, quadratic windows, $\eta_1 = 0.5$, $\eta_2 = 1.5$.]{Same as in Figure \ref{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset1}, but with $\eta_1 = 0.5$ and $\eta_2 = 1.5$.}
	\label{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset2}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, quadratic windows, $\eta_1 = 0.5$, $\eta_2 = 1.5$.]{Comparison of choosing $\lambda$ adaptively or not for fast NL-means. A $21 \times 21$ search window, a $7 \times 7$ similarity window and $\eta_1 = 0.5$, $\eta_2 = 1.5$ is used. This table corresponds to Figure \ref{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset2}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		$\lambda$ & Adaptive & Precision & Recall & F-measure \\
		\hline
		Original & -- & 0.213 & 0.221 & 0.217 \\
		\hline
		1 & no & 0.173 & 0.179 & 0.176 \\
		1 & yes & 0.256 & 0.266 & 0.261 \\
		\hline
		5 & no & 0.117 & 0.122 & 0.119 \\
		5 & yes & 0.136 & 0.141 & 0.138 \\
		\hline
		10 & no & 0.108 & 0.112 & 0.110 \\
		10 & yes & 0.120 & 0.125 & 0.123
	\end{tabular}
	\label{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset2.table}
\end{table}

\paragraph*{Horizontal directional preference.} We finally look at the same setting as in the experiments described before but use a rectangular $7 \times 3$ similarity window, see Figures \ref{experiments.chord.direction.nonadaptive_adaptive.fast.etaset1} and \ref{experiments.chord.direction.nonadaptive_adaptive.fast.etaset2} with the corresponding Tables \ref{experiments.chord.direction.nonadaptive_adaptive.fast.etaset1.table} and \ref{experiments.chord.direction.nonadaptive_adaptive.fast.etaset2.table}.

For a rectangular window, we observe exactly the opposite in comparison to a quadratic one. For the combination of $\eta_1 = 0.5$ and $\eta_2 = 1.5$, the results are advantageous in comparison to the ones computed with $\eta_1 = 0.9$ and $\eta_2 = 1.1$. The reason for this issue can be explained with the following intuition.

So far, we have seen several ingredients that influence the outcome of the NL-means algorithm: the choice of $\lambda$ which controls the weighting, the change of the similarity window to only perform inpainting with a preference for a certain direction and the \emph{fast} NL-means algorithm that only uses the weighting function under certain circumstances which is dependent -- among other conditions -- on the choice of $\eta_1$ and $\eta_2$. Making the interval given in \eqref{nlmeans_avg_grayvalue_frac} bigger also means to include more pixels in the computation of the weighted averaging since the algorithm is more tolerant concerning deviations from the average grayvalue of the two windows that are compared. In some cases, narrowing the interval in \eqref{nlmeans_avg_grayvalue_frac} and using a rectangular similarity window at the same time results in too few information for the inpainting step. On the other hand, including too much information might also not be useful for the denoising process.%, compare Tables \ref{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset1.table} and \ref{experiments.chord.nodirection.nonadaptive_adaptive.fast.etaset2.table}.

This behavior described above cannot be seen with every testfile. It can also be observed that for some input data the outcome is enhanced more for $\eta_1 = 0.9$ and $\eta_2 = 1.1$. Since the result in this example does not differ much for the settings $\eta_1 = 0.9, \eta_2 = 1.1$ and $\eta_1 = 0.5, \eta_2 = 1.5$, we stick to $\eta_1 = 0.9$ and $\eta_2 = 1.1$.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n12.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_1_1_n_09_11.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_1_1_y_09_11.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_1_5_n_09_11.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_1_5_y_09_11.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_1_10_n_09_11.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_1_10_y_09_11.eps}
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, rectangular window, $\eta_1 = 0.9$, $\eta_2 = 1.1$.]{Comparison of choosing $\lambda$ adaptively or not for fast NL-means using the testfile ``Tell Me Why'' by The Beatles (number 12 in Table \ref{appendix.table_chord_files}). All spectrograms were computed using a $21 \times 21$ search window, a $7 \times 3$ similarity window, $\eta_1 = 0.9$ and $\eta_2 = 1.1$. \textbf{Top:} Original spectrogram. \textbf{Left:} $\lambda$ is not chosen adaptively. \textbf{Right:} $\lambda$ is chosen adaptively. \textbf{Middle to bottom:} $\lambda = 1, 5, 10$.}
	\label{experiments.chord.direction.nonadaptive_adaptive.fast.etaset1}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, rectangular window, $\eta_1 = 0.9$, $\eta_2 = 1.1$.]{Comparison of choosing $\lambda$ adaptively or not for fast NL-means with $\eta_1 = 0.9$ and $\eta_2 = 1.1$ where a $21 \times 21$ search window and a $7 \times 3$ similarity window is used. This table corresponds to Figure \ref{experiments.chord.direction.nonadaptive_adaptive.fast.etaset1}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		$\lambda$ & Adaptive & Precision & Recall & F-measure \\
		\hline
		Original & -- & 0.213 & 0.221 & 0.217 \\
		\hline
		1 & no & 0.253 & 0.263 & 0.258 \\
		1 & yes & 0.235 & 0.244 & 0.239 \\
		\hline
		5 & no & 0.231 & 0.240 & 0.236 \\
		5 & yes & 0.238 & 0.247 & 0.242 \\
		\hline
		10 & no & 0.228 & 0.237 & 0.233 \\
		10 & yes & 0.235 & 0.244 & 0.239
	\end{tabular}
	\label{experiments.chord.direction.nonadaptive_adaptive.fast.etaset1.table}
\end{table}

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n12.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_1_1_n_05_15.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_1_1_y_05_15.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_1_5_n_05_15.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_1_5_y_05_15.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_1_10_n_05_15.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/nlmeansfast/n12_10_10_3_1_10_y_05_15.eps}
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, rectangular window, $\eta_1 = 0.5$, $\eta_2 = 1.5$.]{Same as in Figure \ref{experiments.chord.direction.nonadaptive_adaptive.fast.etaset1}, but with $\eta_1 = 0.5$ and $\eta_2 = 1.5$.}
	\label{experiments.chord.direction.nonadaptive_adaptive.fast.etaset2}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, rectangular window, $\eta_1 = 0.5$, $\eta_2 = 1.5$.]{Comparison of choosing $\lambda$ adaptively or not for fast NL-means with $\eta_1 = 0.5$ and $\eta_2 = 1.5$. Furthermore, a $21 \times 21$ search window and a $7 \times 3$ similarity window is used. This table corresponds to Figure \ref{experiments.chord.direction.nonadaptive_adaptive.fast.etaset2}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		$\lambda$ & Adaptive & Precision & Recall & F-measure \\
		\hline
		Original & -- & 0.213 & 0.221 & 0.217 \\
		\hline
		1 & no & 0.238 & 0.247 & 0.242 \\
		1 & yes & 0.225 & 0.234 & 0.230 \\
		\hline
		5 & no & 0.247 & 0.256 & 0.252 \\
		5 & yes & 0.241 & 0.250 & 0.245 \\
		\hline
		10 & no & 0.247 & 0.256 & 0.252 \\
		10 & yes & 0.247 & 0.256 & 0.252
	\end{tabular}
	\label{experiments.chord.direction.nonadaptive_adaptive.fast.etaset2.table}
\end{table}

%%% CHORD EXPERIMENTS: DISCUSSION/SUMMARY %%%
\clearpage
\section{Discussion and Summary}\label{chord_experiments.summary}
This section concludes the experiments on chord recognition\index{chord recognition} by testing a standard parameter set for EED, CED, skeletonization, NL-means and fast NL-means on all the testfiles. This is useful since in real-world examples, a ground truth is generally not available. Thus, having a reasonable starting point is crucial. Furthermore, standardized parameter settings are important for the automated application of the methods. Since the impact of the different methods is visually demonstrated in the previous sections, only the tables giving precision\index{precision}, recall\index{recall} and F-measure\index{F-measure} are proposed here.

For the computations, the following parameter sets are applied and preference for horizontal structures is always used except for skeletonization.
\begin{itemize}
	\item EED: \\
		$\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$, $t = 100$.
	\item CED: \\
		$C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 0.01$, $\tau = 0.25$, $t = 1000$.
	\item Skeletonization: \\
		$s_x = 0.1$, $s_y = 0.1$, threshold = $0.001$.
	\item NL-means: \\
		$21 \times 21$ search window, $7 \times 3$ similarity window, $\lambda = 1$, with adaptive weighting.
	\item Fast NL-means: \\
		$21 \times 21$ search window, $7 \times 3$ similarity window, $\eta_1 = 0.9$, $\eta_2 = 1.1$, $\lambda = 1$, with adaptive weighting.
\end{itemize}

\paragraph*{Performance comparison across all approaches.} In the following, we state the results of the different methods applied to all testfiles with the use of precision, recall and F-measure. The quality of the algorithmic treatment is given for each file in Tables \ref{experiments.chord.file1.table} to \ref{experiments.chord.file13.table}.

As a general trend, we note that diffusion processes outperform skeletonization and NL-means. The testfiles ``Helter Skelter'' and ``Tell Me Why'' are the only exceptions to this finding, compare Tables \ref{experiments.chord.file7.table} and \ref{experiments.chord.file12.table}, where none of the diffusion processes could beat one of the other algorithms. On ``Cry Baby Cry'' and ``You Can't Do That'' (see Tables \ref{experiments.chord.file6.table} and \ref{experiments.chord.file13.table}) at least one of the diffusion processes was superior to all other algorithms. Furthermore, we find that skeletonization beats both NL-means approaches on several testfiles (numbers 1, 3, 4, 6, 7 and 13). In addition, there exist cases where skeletonization is superior to NL-means, but not fast NL-means, see Tables \ref{experiments.chord.file2.table}, \ref{experiments.chord.file5.table}, \ref{experiments.chord.file10.table} and \ref{experiments.chord.file11.table}, and on one of the testfiles, the application of skeletonization gives a higher F-measure compared to fast NL-means, but not NL-means (Table \ref{experiments.chord.file8.table}).
%Of course, this observation strongly depends on the chosen parameters, compare Tables \ref{experiments.chord.file7.table} and \ref{experiments.chord.file12.table}.

Within the class of NL-means, we notice that the fast NL-means algorithm is superior to the NL-means algorithm in almost every case. This observation supports the theory that too much information leads to a significant error in the weighted averaging. The only exception is the file ``I Am The Walrus'', compare Table \ref{experiments.chord.file8.table}. In general, the fast NL-means algorithm uses the weighting function in comparison to NL-means less often. For this concrete example this may result in too few data for the inpainting step since the spectrogram is sparse in the first half (0 to about 15 seconds). Thus, every time the weighting function is used, it consists of few contributions from the inpainting neighborhood, i.e.\ the similarity window.

\paragraph*{Runtime considerations.} As already stated earlier, the runtime of the algorithms is not the main concern of this thesis. Nevertheless, let us give for the sake of completeness one example. Table \ref{experiments.chord.file5.runtime.table} shows the runtime of the different algorithms on the file ``Back In The USSR'' (number 5 in Table \ref{appendix.table_chord_files}). Let us mention that NL-means still needs a high runtime even though the search space is restricted by the use of a quadratic search window.

\paragraph*{Combination of different filters.} It is also natural to combine different methods. Exemplarily, we investigate skeletonization as a preprocessing as well as a postprocessing step to diffusion on the testfile ``Cry Baby Cry''. Since edge-enhancing diffusion and coherence-enhancing diffusion lead to approximately the same result under similar parameter settings, we use edge-enhancing diffusion here. For the diffusion process as well as for the skeletonization we stick to the parameters proposed earlier.

The spectrograms of this experiment where skeletonization is applied before and after edge-enhancing diffusion are given in Figure \ref{experiments.chord.skeletonization.preprocessing_postprocessing} and the corresponding values for precision, recall and F-measure in Table \ref{experiments.chord.skeletonization.preprocessing_postprocessing.table}.

Let us have a look at the spectrograms on the bottom of Figure \ref{experiments.chord.skeletonization.preprocessing_postprocessing}. The visual quality of both is different. The one on the right is very blurry which is not the case for the spectrogram on the left. Only judging the visual quality, we probably would say that the spectrogram on the left is advantageous compared to the right one. Recalling our observation from Section \ref{chord_experiments.skeletonization}, we can explain why this is not the case (see also Table \ref{experiments.chord.skeletonization.preprocessing_postprocessing.table}). Skeletonization makes the edges -- i.e.\ frequency contributions -- thinner. This also removes some important information about the frequency composition that cannot be repaired by applying a diffusion process afterwards. In contrast, applying the diffusion before skeletonization is desired since the diffusion process enhances structures that should not be enhanced and skeletonization works as a sort of correction step for this issue.

\paragraph*{Limitations of chord recognition.} To conclude the experiments on chord recognition\index{chord recognition}, we want to mention an interesting point. In every information retrieval task it can happen that the feature extraction might extract something wrong. In the following example, the feature extraction is -- informally speaking -- both right and wrong: right due to the input data (the spectrogram) and wrong due to acoustical judgement. To illustrate this, we consider ``And I Love Her'' by The Beatles (number 3 in Table \ref{appendix.table_chord_files}).
%Performing edge-enhancing diffusion using $\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$ and $t = 80$ with directional preference for horizontal structures removes some noise.
Here it is convenient to look additionally at the chroma-time representation and chord annotation. Figure \ref{experiments.chord.interesting_result} gives both the original data as well as the processed data. Let us look at the bottom of the figure. This plot shows in a color coding which chords are present at which time. Gray stands for the chords that are annotated, red for the retrieved chords. If both match, the color is white. The numbers on the y-axis identify the chords where numbers from 1 to 12 stand for C Major to B Major and numbers from 13 to 24 for C Minor to B Minor. Let us look at the time frame 0 to 5 seconds on the bottom right representation in Figure \ref{experiments.chord.interesting_result}. The feature extraction detected F$^\sharp$ Major (chord number 7) where the annotation tells that it should be F$^\sharp$ Minor (chord number 19). This is one of the worst classification errors that can happen. Please note that this misclassification is not introduced, but slightly enhanced by the diffusion process, see bottom row of Figure \ref{experiments.chord.interesting_result}. The question is why chord recognition severely fails here.

To answer this, we have to introduce a little bit of music theory. The chord F$^\sharp$ Major consists of the notes F$^\sharp$, A$^\sharp$ and C$^\sharp$ where F$^\sharp$ Minor consists of the notes F$^\sharp$, A and C$^\sharp$. The only difference is the note in the middle. This must mean that the feature extraction detected an A$^\sharp$ instead of A. And in fact, if we have a look at the chroma-time representation of the processed data (middle right in Figure \ref{experiments.chord.interesting_result}) at the point in time where the note F$^\sharp$ is almost represented in white, we notice that the note A$^\sharp$ has a brighter color than A meaning that it is more intense at this point in time. After listening to the testfile, the answer why chord recognition fails on this file is obvious: at exactly this point, the percussive elements start. The percussion contributes to the frequencies corresponding to A$^\sharp$ which makes it more intense than A. Thus, the feature extraction doesn't recognize the note A as important as the note A$^\sharp$ meaning that we do not end up with the notes F$^\sharp$, A, C$^\sharp$ as the most important notes at this time but with F$^\sharp$, A$^\sharp$, C$^\sharp$. Thus, the chord recognition algorithm concludes that the chord F$^\sharp$ Major instead of F$^\sharp$ Minor is played.

\paragraph*{Final remark.} As we have seen, image processing methods are able to significantly enhance the spectrograms that were considered in this chapter and thus can contribute fruitfully to the success of chord recognition.

\clearpage
\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for ``All My Loving''.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``All My Loving'' by The Beatles (file number 1 in Table \ref{appendix.table_chord_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.413 & 0.426 & 0.419 \\
		\hline
		EED & 0.557 & 0.574 & 0.566 \\
		CED & 0.469 & 0.483 & 0.476 \\
		\hline
		Skeletonization & 0.413 & 0.426 & 0.419 \\
		\hline
		NL-means & 0.402 & 0.414 & 0.408 \\
		Fast NL-means & 0.402 & 0.414 & 0.408
	\end{tabular}
	\label{experiments.chord.file1.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for ``All You Need Is Love''.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``All You Need Is Love'' by The Beatles (file number 2 in Table \ref{appendix.table_chord_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.412 & 0.416 & 0.414 \\
		\hline
		EED & 0.902 & 0.910 & 0.906 \\
		CED & 0.569 & 0.575 & 0.572 \\
		\hline
		Skeletonization & 0.446 & 0.450 & 0.448 \\
		\hline
		NL-means & 0.308 & 0.311 & 0.309 \\
		Fast NL-means & 0.468 & 0.472 & 0.470
	\end{tabular}
	\label{experiments.chord.file2.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for ``And I Love Her''.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``And I Love Her'' by The Beatles (file number 3 in Table \ref{appendix.table_chord_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.461 & 0.484 & 0.472 \\
		\hline
		EED & 0.594 & 0.623 & 0.609 \\
		CED & 0.480 & 0.503 & 0.491 \\
		\hline
		Skeletonization & 0.458 & 0.481 & 0.469 \\
		\hline
		NL-means & 0.421 & 0.442 & 0.431 \\
		Fast NL-means & 0.424 & 0.445 & 0.434
	\end{tabular}
	\label{experiments.chord.file3.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for ``Baby's In Black''.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``Baby's In Black'' by The Beatles (file number 4 in Table \ref{appendix.table_chord_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.475 & 0.482 & 0.479 \\
		\hline
		EED & 0.695 & 0.705 & 0.700 \\
		CED & 0.562 & 0.570 & 0.566 \\
		\hline
		Skeletonization & 0.537 & 0.545 & 0.541 \\
		\hline
		NL-means & 0.389 & 0.395 & 0.392 \\
		Fast NL-means & 0.456 & 0.463 & 0.459
	\end{tabular}
	\label{experiments.chord.file4.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for ``Back In The USSR''.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``Back In The USSR'' by The Beatles (file number 5 in Table \ref{appendix.table_chord_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.491 & 0.495 & 0.493 \\
		\hline
		EED & 0.613 & 0.619 & 0.616 \\
		CED & 0.564 & 0.570 & 0.567 \\
		\hline
		Skeletonization & 0.500 & 0.505 & 0.502 \\
		\hline
		NL-means & 0.402 & 0.406 & 0.404 \\
		Fast NL-means & 0.525 & 0.529 & 0.527
	\end{tabular}
	\label{experiments.chord.file5.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for ``Cry Baby Cry''.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``Cry Baby Cry'' by The Beatles (file number 6 in Table \ref{appendix.table_chord_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.335 & 0.335 & 0.335 \\
		\hline
		EED & 0.480 & 0.480 & 0.480 \\
		CED & 0.369 & 0.369 & 0.369 \\
		\hline
		Skeletonization & 0.394 & 0.394 & 0.394 \\
		\hline
		NL-means & 0.305 & 0.305 & 0.305 \\
		Fast NL-means & 0.351 & 0.351 & 0.351
	\end{tabular}
	\label{experiments.chord.file6.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for ``Helter Skelter''.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``Helter Skelter'' by The Beatles (file number 7 in Table \ref{appendix.table_chord_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.141 & 0.142 & 0.142 \\
		\hline
		EED & 0.144 & 0.145 & 0.145 \\
		CED & 0.154 & 0.155 & 0.154 \\
		\hline
		Skeletonization & 0.232 & 0.233 & 0.233 \\
		\hline
		NL-means & 0.110 & 0.110 & 0.110 \\
		Fast NL-means & 0.179 & 0.180 & 0.179
	\end{tabular}
	\label{experiments.chord.file7.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for ``I Am The Walrus''.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``I Am The Walrus'' by The Beatles (file number 8 in Table \ref{appendix.table_chord_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.456 & 0.464 & 0.460 \\
		\hline
		EED & 0.562 & 0.573 & 0.567 \\
		CED & 0.520 & 0.529 & 0.525 \\
		\hline
		Skeletonization & 0.483 & 0.492 & 0.488 \\
		\hline
		NL-means & 0.502 & 0.511 & 0.506 \\
		Fast NL-means & 0.480 & 0.489 & 0.485
	\end{tabular}
	\label{experiments.chord.file8.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for ``I'm Only Sleeping''.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``I'm Only Sleeping'' by The Beatles (file number 9 in Table \ref{appendix.table_chord_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.352 & 0.357 & 0.354 \\
		\hline
		EED & 0.344 & 0.349 & 0.346 \\
		CED & 0.386 & 0.391 & 0.389 \\
		\hline
		Skeletonization & 0.315 & 0.319 & 0.317 \\
		\hline
		NL-means & 0.323 & 0.327 & 0.325 \\
		Fast NL-means & 0.333 & 0.338 & 0.336
	\end{tabular}
	\label{experiments.chord.file9.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for ``Let It Be''.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``Let It Be'' by The Beatles (file number 10 in Table \ref{appendix.table_chord_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.496 & 0.496 & 0.496 \\
		\hline
		EED & 0.585 & 0.585 & 0.585 \\
		CED & 0.606 & 0.606 & 0.606 \\
		\hline
		Skeletonization & 0.504 & 0.504 & 0.504 \\
		\hline
		NL-means & 0.430 & 0.430 & 0.430 \\
		Fast NL-means & 0.532 & 0.532 & 0.532
	\end{tabular}
	\label{experiments.chord.file10.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for ``Savoy Truffle''.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``Savoy Truffle'' by The Beatles (file number 11 in Table \ref{appendix.table_chord_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.185 & 0.195 & 0.190 \\
		\hline
		EED & 0.370 & 0.390 & 0.380 \\
		CED & 0.247 & 0.260 & 0.253 \\
		\hline
		Skeletonization & 0.222 & 0.234 & 0.228 \\
		\hline
		NL-means & 0.198 & 0.208 & 0.203 \\
		Fast NL-means & 0.244 & 0.256 & 0.250
	\end{tabular}
	\label{experiments.chord.file11.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for ``Tell Me Why''.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``Tell Me Why'' by The Beatles (file number 12 in Table \ref{appendix.table_chord_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.213 & 0.221 & 0.217 \\
		\hline
		EED & 0.142 & 0.147 & 0.145 \\
		CED & 0.204 & 0.212 & 0.208 \\
		\hline
		Skeletonization & 0.207 & 0.215 & 0.211 \\
		\hline
		NL-means & 0.216 & 0.224 & 0.220 \\
		Fast NL-means & 0.235 & 0.244 & 0.239
	\end{tabular}
	\label{experiments.chord.file12.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for ``You Can't Do That''.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``You Can't Do That'' by The Beatles (file number 13 in Table \ref{appendix.table_chord_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.207 & 0.211 & 0.209 \\
		\hline
		EED & 0.247 & 0.252 & 0.249 \\
		CED & 0.262 & 0.267 & 0.265 \\
		\hline
		Skeletonization & 0.253 & 0.258 & 0.255 \\
		\hline
		NL-means & 0.148 & 0.151 & 0.150 \\
		Fast NL-means & 0.210 & 0.214 & 0.212
	\end{tabular}
	\label{experiments.chord.file13.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of algorithm runtime for ``Back In The USSR''.]{Comparison of algorithm runtime for ``Back In The USSR'' by The Beatles.}
	\ \\[0.3cm]
	\begin{tabular}{cr}
		Method & Runtime [s] \\
		\hline
		EED & 1.734504 \\
		CED & 49.354818 \\
		\hline
		Skeletonization & 75.949182 \\
		\hline
		NL-means & 4757.615534 \\
		Fast NL-means & 270.311324
	\end{tabular}
	\label{experiments.chord.file5.runtime.table}
\end{table}

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n6.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/chains/n6_eed_skel.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/chains/n6_skel_eed.eps}
	\caption[Skeletonization as pre-/postprocessing step for diffusion.]{Skeletonization as preprocessing or postprocessing step for diffusion. \textbf{Top:} Original spectrogram of ``Cry Baby Cry'' by The Beatles (number 6 in Table \ref{appendix.table_chord_files}). \textbf{Bottom left:} Application of a diffusion process followed by a skeletonization. \textbf{Bottom right:} Application of a skeletonization followed by a diffusion process.}
	\label{experiments.chord.skeletonization.preprocessing_postprocessing}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[Skeletonization as pre-/postprocessing step for diffusion.]{Skeletonization as preprocessing or postprocessing step for diffusion. This table gives the corresponding values for precision, recall and F-measure for the spectrograms given in Figure \ref{experiments.chord.skeletonization.preprocessing_postprocessing}.}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Application chain & Precision & Recall & F-measure \\
		\hline
		Original & 0.335 & 0.335 & 0.335 \\
		EED $\rightarrow$ Skeletonization & 0.486 & 0.486 & 0.486 \\
		Skeletonization $\rightarrow$ EED & 0.572 & 0.572 & 0.572
	\end{tabular}
	\label{experiments.chord.skeletonization.preprocessing_postprocessing.table}
\end{table}

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n3.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/eed/n3_80_0_1.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n3_chroma.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/eed/n3_80_0_1_chroma.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/chord/original/n3_annotation.eps}
	\includegraphics[scale=0.42]{figures/experiments/chord/eed/n3_80_0_1_annotation.eps}
	\caption[Example where chord recognition is fooled by percussion.]{Example where chord recognition is fooled by percussion. \textbf{Top left:} Original spectrogram of ``And I Love Her'' by The Beatles (number 3 in Table \ref{appendix.table_chord_files}). \textbf{Top right:} Edge-enhancing diffusion with $\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$, $t = 80$ and directional preference for horizontal structures. \textbf{Middle left:} Chroma-time representation of the original spectrogram. \textbf{Middle right:} Corresponding chroma-time representation of the modified spectrogram. \textbf{Bottom left:} Representation of the original data that shows the chords that are annotated and the chords that are retrieved in one image. Numbers from 1 to 12 represent the chords from C Major to B Major and from 13 to 24 the chords C Minor to B Minor. Gray color denotes annotated chords, red color retrieved chords. If both match, the color turns white. \textbf{Bottom right:} Same representation of the processed data.}
	\label{experiments.chord.interesting_result}
\end{figure}

%%%%%%%%%%%%%%%%%%%  %%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%  %%%%%%%%%%%%%%%%%%%
%%% NEW CHAPTER %%%  %%% NEW CHAPTER %%% %%% NEW CHAPTER %%%  %%% NEW CHAPTER %%%
%%%%%%%%%%%%%%%%%%%  %%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%  %%%%%%%%%%%%%%%%%%%

\pagestyle{fancy}
\cleardoublepage\chapter{Experiments in Onset Detection}\label{onset_experiments}
\Quote{Being able to quit things that don't work is integral to being a winner.}{Tim Ferriss}

\noindent This chapter describes experiments for the task of onset detection\index{onset detection} where we are interested in vertical structures. The same experimental setup as described in Chapter \ref{chord_experiments} is used. Again, the quality of the experimental results is judged by using the concepts introduced in Section \ref{tasks.quality_measurement}.

As already mentioned in Section \ref{tasks.onset_detection}, onset detection is not an easy task. While it seems well-defined at first glance, it is difficult to locate the starting point of a musical note.

For the following experiments, we use the testfiles listed in Table \ref{appendix.table_onset_files}. The testfiles 1 to 8 are excerpts manually annotated by Peter Grosche and Meinard M{\"u}ller (see \cite{grosche09-2}). The testfiles 9 to 23 are taken from \cite{rwc}, except for 10, 21 and 22 that are not publicly available, see also \cite{leveau04}.

%%% ONSET EXPERIMENTS: DIFFUSION %%%
\section{Diffusion\index{diffusion}}\label{onset_experiments.diffusion}

This section deals with diffusion\index{diffusion} processes that are performed on the spectrograms\index{spectrogram} for the task of onset detection\index{onset detection}. Since we are interested in vertical structures, we first compare the results of diffusion\index{diffusion} processes with and without a directional preference for a fixed number of iteration steps. In a second experiment, we investigate the behavior of the algorithm for different numbers of iterations where $d_\beta$ is set to 1 and $d_\alpha$ is set to 0 which means that we prefer vertical structures.

%%% ONSET EXPERIMENTS: EED %%%
\subsection{Edge-Enhancing Diffusion\index{diffusion!edge-enhancing}}

\paragraph*{Vertical vs. no directional preference.} We start by investigating edge-enhancing diffusion on the onset detection problem for the same number of iterations but once with and once without preference for vertical directions. Figure \ref{experiments.onset.eed_preference} depicts the spectrograms of the ``Happy Birthday'' testfile performed by a choir accompanied by a piano. The spectrogram after applying EED without directional preference is shown on the bottom left where the result of applying EED with vertical preference is given on the bottom right. The spectrogram on the bottom left shows that diffusion happened also in horizontal direction resulting in the fact that onsets cannot be clearly spotted. Comparing the values for the F-measure in Table \ref{experiments.onset.eed_preference.table}, we observe that the diffusion process without vertical preference slightly enhances the result where the process that diffuses only in vertical direction improves the quality of the computation even further. The investigation of the spectrogram on the bottom right in Figure \ref{experiments.onset.eed_preference} supports this since the structures are sharp and only enhanced in vertical direction. The results given in Table \ref{experiments.onset.eed_preference.table} coincide with the intuition that a diffusion performed equally in every direction, i.e.\ a blurring also in the time domain, does not give optimal results with respect to the onset detection problem.

In this context, we also want to recall an important aspect concerning the result given by the quality measure. The diffusion process without a directional preference gives a higher precision than the diffusion process with a directional preference but this comes at the cost of a decrease in recall. The result for the process with (only) vertical diffusion improves not just the precision but also the recall with respect to the original data. This example shows that we always have to consider the quantities precision and recall together, compare Section \ref{tasks.quality_measurement}.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n4.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/eed/n4_20_1_1.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/eed/n4_20_1_0.eps}
	\caption[EED, comparison between directional preference and no preference.]{Spectrograms of ``Happy Birthday'' (number 4 in Table \ref{appendix.table_onset_files}). \textbf{Top:} Original spectrogram. \textbf{Bottom left:} Spectrogram after diffusion with EED and $\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$, $t = 20$ where no direction is preferred. \textbf{Bottom right:} Spectrogram after diffusion with the same setting, but directional preference for vertical diffusion.}
	\label{experiments.onset.eed_preference}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[EED, comparison between directional preference and no preference.]{Comparison of precision\index{precision}, recall\index{recall} and F-measure\index{F-measure} for EED with and without directional preference. Edge-enhancing diffusion was performed with $\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$ and $t = 20$. The values correspond to the processing depicted in Figure \ref{experiments.onset.eed_preference}.}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Setting & Precision & Recall & F-Measure \\
		\hline
		Original & 0.383 & 0.692 & 0.493 \\
		No preference & 0.571 & 0.462 & 0.511 \\
		Vertical preference & 0.455 & 0.769 & 0.571
	\end{tabular}
	\label{experiments.onset.eed_preference.table}
\end{table}

\paragraph*{Increasing the number of iterations.} As we have seen in Table \ref{experiments.onset.eed_preference.table}, diffusion with vertical preference gives improved results in comparison to unbiased anisotropic diffusion. We proceed by fixing the diffusion preference to vertical direction and investigate how increasing the number of iterations influences the result. Figure \ref{experiments.onset.eed_iterations} shows again the ``Happy Birthday'' testfile. The spectrograms are processed by edge-enhancing diffusion with different numbers of iterations. The F-measure given in Table \ref{experiments.onset.eed_iterations.table} suggests that the result for 100 iterations is an outlier and the more iterations we perform the more the final result is enhanced. Considering all the quantities of the quality measures, we see that the high precision and the high F-measure for 200 iterations comes with a low recall which is not desired. For 20 iterations we observe an improved result since we have an increase in precision as well as in recall in comparison to the original data.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n4.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/eed/n4_20_1_0.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/eed/n4_100_1_0.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/eed/n4_200_1_0.eps}
	\caption[EED, comparison between different numbers of iterations.]{Spectrograms of ``Happy Birthday'' (number 4 in Table \ref{appendix.table_onset_files}). All diffusion processes are performed with a preference for vertical structures. \textbf{Top left:} Original spectrogram. \textbf{Top right:} Spectrogram after diffusion with EED and $\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$ and $t = 20$. \textbf{Bottom left:} $t = 100$. \textbf{Bottom right:} $t = 200$.}
	\label{experiments.onset.eed_iterations}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[EED, comparison between different numbers of iterations.]{Comparison of precision\index{precision}, recall\index{recall} and F-measure\index{F-measure} for EED with different numbers of iterations and directional preference for vertical structures where $\lambda = 1$, $\sigma = 0.5$ and $\tau = 0.25$. The values correspond to the processing depicted in Figure \ref{experiments.onset.eed_iterations}.}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Iterations & Precision & Recall & F-Measure \\
		\hline
		0 & 0.383 & 0.692 & 0.493 \\
		20 & 0.455 & 0.769 & 0.571 \\
		100 & 0.442 & 0.731 & 0.551 \\
		200 & 0.487 & 0.731 & 0.585
	\end{tabular}
	\label{experiments.onset.eed_iterations.table}
\end{table}

%%% ONSET EXPERIMENTS: CED %%%
\subsection{Coherence-Enhancing Diffusion\index{diffusion!coherence-enhancing}}

\paragraph*{Vertical vs. no directional preference.} In the context of coherence-enhancing diffusion, we begin again by investigating the impact of diffusion only in vertical direction in comparison to unbiased anisotropic diffusion. Figure \ref{experiments.onset.ced_preference} shows the spectrograms of the testfile ``guitar3'' after processing the original spectrogram with coherence-enhancing diffusion with and without directional preference. In general, we have to note that the spectrogram of this testfile is very dense. Thus, investigating the resulting spectrograms carefully is necessary to spot the direction of the diffusion process. Nevertheless, having a look at the frequency contributions (in yellow color) slightly below 5000, we see that they are also blurred in horizontal direction in the spectrogram on the bottom left where these contributions are not horizontally extended in the version on the bottom right. Alternatively, we can investigate the frequencies that are above 6000. In the original spectrogram there are thin vertical structures. These are not only preserved but also enhanced by the diffusion algorithm with preference in vertical direction where these structures are blurred away for the diffusion process that performs unbiased anisotropic diffusion, compare the spectrograms on the bottom right and bottom left in Figure \ref{experiments.onset.ced_preference}, respectively. Table \ref{experiments.onset.ced_preference.table} also shows that the setting with vertical preference is superior to the setting without directional preference.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n15.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/ced/n15_100_1_1.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/ced/n15_100_1_0.eps}
	\caption[CED, comparison between directional preference and no preference.]{Spectrograms of ``guitar3'' (number 15 in Table \ref{appendix.table_onset_files}). \textbf{Top:} Original spectrogram. \textbf{Bottom left:} Spectrogram after diffusion with CED and $C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 0.1$, $\tau = 0.25$, $t = 100$ where no direction is preferred. \textbf{Bottom right:} Spectrogram after diffusion with the same setting, but directional preference for vertical diffusion.}
	\label{experiments.onset.ced_preference}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[CED, comparison between directional preference and no preference.]{Comparison of precision\index{precision}, recall\index{recall} and F-measure\index{F-measure} for CED with and without directional preference where $C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 0.1$, $\tau = 0.25$ and $t = 100$. The values correspond to the processing depicted in Figure \ref{experiments.onset.ced_preference}.}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Setting & Precision & Recall & F-Measure \\
		\hline
		Original & 1.000 & 0.828 & 0.906 \\
		No preference & 1.000 & 0.034 & 0.067 \\
		Vertical preference & 1.000 & 0.845 & 0.916
	\end{tabular}
	\label{experiments.onset.ced_preference.table}
\end{table}

\paragraph*{Increasing the number of iterations.} We proceed by investigating how an increase in the number of iterations influences the result of the coherence-enhancing diffusion where we fix the process to only diffuse in vertical direction, see Figure \ref{experiments.onset.ced_iterations}. As already mentioned before, the spectrogram of the ``guitar3'' testfile is very dense. Performing a large number of iterations introduces an even higher density which does not enhance the overall quality of the resulting spectrogram, see Table \ref{experiments.onset.ced_iterations.table}. For 500 or 1000 iterations, the recall is lower than without any processing which shows that the task of onset detection is more difficult for these spectrograms. In contrast, we notice that for 100 iterations, the result is slightly improved. In this case, information about the structure is still preserved where for 500 or 1000 iterations, this information is blurred completely, compare e.g.\ the bottom of the corresponding spectrograms (in bright yellow).

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n15.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/ced/n15_100_1_0.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/ced/n15_500_1_0.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/ced/n15_1000_1_0.eps}
	\caption[CED, comparison between different numbers of iterations.]{Spectrograms of ``guitar3'' (number 15 in Table \ref{appendix.table_onset_files}). All diffusion processes are performed with a preference for vertical structures. \textbf{Top left:} Original spectrogram. \textbf{Top right:} Spectrogram after diffusion with CED and $C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 0.1$, $\tau = 0.25$, $t = 100$. \textbf{Bottom left:} $t = 500$. \textbf{Bottom right:} $t = 1000$.}
	\label{experiments.onset.ced_iterations}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[CED, comparison between different numbers of iterations.]{Comparison of precision\index{precision}, recall\index{recall} and F-measure\index{F-measure} for CED ($C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 0.1$ and $\tau = 0.25$) with vertical preference and different numbers of iterations. The values correspond to the processing depicted in Figure \ref{experiments.onset.ced_iterations}.}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Iterations & Precision & Recall & F-Measure \\
		\hline
		0 & 1.000 & 0.828 & 0.906 \\
		100 & 1.000 & 0.845 & 0.916 \\
		500 & 1.000 & 0.810 & 0.895 \\
		1000 & 1.000 & 0.810 & 0.895
	\end{tabular}
	\label{experiments.onset.ced_iterations.table}
\end{table}

%%% ONSET EXPERIMENTS: EED vs. CED %%%
\clearpage
\subsection{Comparison of EED and CED}\label{onset_experiments.eed_vs_ced}
We have seen in the previous sections that edge-enhancing diffusion and coherence-enhancing diffusion behave similarly for onset detection: first, a process with directional preference is superior to a process without directional preference and second, the more a process diffuses, the worse gets the result. Like we did in Chapter \ref{chord_experiments}, we compare EED and CED head to head while using similar parameter settings. Please remember that $\alpha$ in the CED algorithm is sort of a damping of the diffusion process. To allow a similar behavior of both diffusion algorithms for the same number of iterations, we have to set $\alpha = 1$ for coherence-enhancing diffusion.
% To perform exactly the same number of iterations for EED and CED, we have to set $\alpha = 1$ for coherence-enhancing diffusion.

The original spectrogram of the testfile Op.\ 100 No.\ 2 by Friedrich Burgm{\"u}ller is given on the top of Figure \ref{experiments.onset.eed_ced_comparison}. Below this, the spectrograms of the data processed with edge-enhancing diffusion and coherence-enhancing diffusion are given side by side. In the middle row, we compare EED and CED where no directional preference is used. Visually, the two results look very similar, but -- according to the quality measure given in Table \ref{experiments.onset.eed_ced_comparison.table} -- the spectrogram computed by the application of EED beats the one computed with CED. Please note that both diffusion algorithms without directional preference worsen the result compared to the original, unprocessed data which coincides with previous findings. The results of the diffusion processes with directional preference for vertical structures is given on the bottom of Figure \ref{experiments.onset.eed_ced_comparison}. Again, the visual judgement reveals no differences between the spectrogram on the bottom left and the one on the bottom right. In this case, also the quality measure given in Table \ref{experiments.onset.eed_ced_comparison.table} shows that the two algorithms perform similarly.

But why do the two diffusion processes act in a similar fashion? To answer this, we refer to Section \ref{chord_experiments.eed_vs_ced}. The explanation already given there for chord recognition also holds for onset detection.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n1.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/eed/n1_50_1_1.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/ced/n1_50_1_1.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/eed/n1_50_1_0.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/ced/n1_50_1_0.eps}
	\caption[Comparison between EED and CED.]{Comparison between EED and CED on the testfile Op.\ 100, No.\ 2 by Friedrich Burgm{\"u}ller (number 1 in Table \ref{appendix.table_onset_files}). \textbf{Top:} Original spectrogram. \textbf{Middle left:} Edge-enhancing diffusion with $\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$, $t = 50$ and no preference for vertical diffusion. \textbf{Bottom left:} Same setting with preference for vertical diffusion. \textbf{Middle right:} Coherence-enhancing diffusion with $C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 1$, $\tau = 0.25$, $t = 50$ and no preference for vertical diffusion. \textbf{Bottom right:} Same parameters, but directional preference for vertical diffusion turned on.}
	\label{experiments.onset.eed_ced_comparison}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[Comparison between EED and CED.]{Comparison between EED with $\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$, $t = 50$ and CED using $C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 1$, $\tau = 0.25$, $t = 50$. The table corresponds to Figure \ref{experiments.onset.eed_ced_comparison}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		Method & Preference & Precision & Recall & F-Measure \\
		\hline
		Original & -- & 0.979 & 0.676 & 0.800 \\
		\hline
		EED & no & 0.941 & 0.235 & 0.376 \\
		CED & no & 0.800 & 0.176 & 0.289 \\
		\hline
		EED & yes & 1.000 & 0.691 & 0.817 \\
		CED & yes & 1.000 & 0.691 & 0.817
	\end{tabular}
	\label{experiments.onset.eed_ced_comparison.table}
\end{table}

%%% ONSET EXPERIMENTS: MORPHOLOGY %%%
\clearpage
\section{Morphology\index{morphology}}\label{onset_experiments.morphology}

Similarly to Section \ref{chord_experiments.morphology}, we start by investigating the basic building blocks of mathematical morphology, i.e.\ dilation, erosion, closing and opening, with a rectangle as (flat) structuring element before proceeding with a parabola (which is a nonflat structuring element) used in parabolic dilation, parabolic erosion and parabolic skeletonization. All methods are performed on the spectrogram of ``rock1'' given in Figure \ref{experiments.onset.morphology.original}.

Please note that we omit the usage of a disk since we have already seen in Section \ref{chord_experiments.morphology} that a (flat) disk is not well suited for the processing of spectrograms.

\begin{figure}[ht]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n20.eps}
	\caption[Spectrogram of ``rock1''.]{Spectrogram of ``rock1'' testfile (number 20 in Table \ref{appendix.table_onset_files}).}
	\label{experiments.onset.morphology.original}
\end{figure}

%%% ONSET EXPERIMENTS: DILATION/EROSION %%%
\subsection{Dilation\index{dilation} and Erosion\index{erosion}}

In the top row of Figure \ref{experiments.onset.morphology.rect}, the results of a dilation (top left) and an erosion (top right) with a rectangle of height 10 and width 1 are depicted. We notice that the dilation process introduces a high amount of blurring in vertical direction in comparison to the original spectrogram, see also Table \ref{experiments.onset.morphology.rect.table}. The problem is that such an enhancement might also give a higher impact to contributions that are no real onsets (e.g.\ noise). In contrast, erosion with the same structuring element reduces small contributions such that in general only strong contributions survive. For onset detection this is advantageous, since only ``true'' onsets may be present. On the other hand, actual onsets with a low contribution in the spectrogram may be eroded away. Considering Table \ref{experiments.onset.morphology.rect.table}, the application of erosion improves the result more than the dilation operation. The precision measure is slightly lower when erosion is applied, but the recall measure is significantly higher which leads to a result with improved overall quality. Please also note that the recall for erosion is not only higher in comparison to dilation but also to the original data.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/morphology/n20_dil_rect_10_1.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/morphology/n20_ero_rect_10_1.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/morphology/n20_close_rect_10_1.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/morphology/n20_open_rect_10_1.eps}
	\caption[Morphological operations on the spectrogram of ``rock1''.]{Morphological operations on the spectrogram of ``rock1'' with a rectangle of height 10 and width 1 as structuring element. \textbf{Top left:} Dilation. \textbf{Top right:} Erosion. \textbf{Bottom left:} Closing. \textbf{Bottom right:} Opening.}
	\label{experiments.onset.morphology.rect}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[Morphological operations on the spectrogram of ``rock1''.]{Morphological operations on the spectrogram of ``rock1'' with a rectangle of height 10 and width 1 as structuring element. See also Figure \ref{experiments.onset.morphology.rect}.}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 1.000 & 0.758 & 0.862 \\
		\hline
		Dilation & 1.000 & 0.710 & 0.830 \\
		Erosion & 0.962 & 0.806 & 0.877 \\
		\hline
		Closing & 0.977 & 0.694 & 0.811 \\
		Opening & 0.940 & 0.758 & 0.839
	\end{tabular}
	\label{experiments.onset.morphology.rect.table}
\end{table}

%%% ONSET EXPERIMENTS: OPENING/CLOSING %%%
\clearpage
\subsection{Opening\index{opening} and Closing\index{closing}}

In this section, we are concerned with the operations opening and closing. The closing operation on the testfile ``rock1'' is depicted in Figure \ref{experiments.onset.morphology.rect} on the bottom left and the opening operation on the bottom right. Please note that the results on the bottom can be achieved by the application of the opposite morphological operation on the spectrograms in the top row. In other words, the spectrogram on the bottom left of Figure \ref{experiments.onset.morphology.rect} can be computed by applying an erosion operation with the same structuring element on the spectrogram depicted on the top left (showing a dilation). Consequently, the spectrogram on the bottom right can be computed by applying a dilation to the spectrogram on the top right.

Let us look at the closing in Figure \ref{experiments.onset.morphology.rect} first. The blurring introduced by the dilation (see top left spectrogram) is reduced. Nevertheless, the dilation operation already introduced a high amount of blurring which cannot be compensated by the followed erosion. The opening operation is depicted in Figure \ref{experiments.onset.morphology.rect} on the bottom right. Here we find that contributions that are almost eroded away (compare top right spectrogram) are restored by the dilation step. This also re-introduces noisy structures that were reduced by the previous erosion operation.

In Chapter \ref{chord_experiments} we have seen that the result of dilation was enhanced by applying erosion afterwards and erosion was enhanced by the successive application of dilation. In other words, the closing operation gave enhancements compared to pure dilation and the opening operation improved the overall result compared to pure erosion. For the task of onset detection, this does not seem to hold. The closing operation worsens the result in comparison to the dilation and the opening in comparison to the erosion, see Table \ref{experiments.onset.morphology.rect.table}.

Please also note that only erosion was able to enhance the overall quality of the testfile.

%%% ONSET EXPERIMENTS: SKELETONIZATION %%%
\subsection{Skeletonization\index{skeletonization}}\label{onset_experiments.skeletonization}

As we did in Section \ref{chord_experiments.skeletonization}, we start by briefly looking at parabolic dilation and parabolic erosion before proceeding to parabolic skeletonization. In addition, we compare the parabolic skeletonization approach as introduced in \ref{modifications.morphology} with the binary skeletonization available in MATLAB.

Figure \ref{experiments.onset.pardil_parero} shows parabolic dilation and parabolic erosion of ``rock1''. In case of dense spectrograms like in this example, parabolic dilation blurs even more than dilation with a rectangle as structuring element. With parabolic erosion, we observe the opposite. Parabolic erosion performs -- in comparison to erosion with a rectangle as structuring element -- stronger thinning of the vertical structures.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/morphology/n20_pardil_01_01.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/morphology/n20_parero_01_01.eps}
	\caption[Parabolic morphological operations on the spectrogram of ``rock1''.]{Parabolic morphological operations on the spectrogram of ``rock1''. \textbf{Left:} Parabolic dilation with $s_x = 0.1$ and $s_y = 0.1$. \textbf{Right:} Parabolic erosion with $s_x = 0.1$ and $s_y = 0.1$.}
	\label{experiments.onset.pardil_parero}
\end{figure}

As already mentioned, MATLAB provides a skeletonization function (\texttt{bwmorph}) which only works on binary images. Thus, the first step is to convert the grayscale image to a binary version. We use again the function \texttt{graythresh} to compute a threshold for \texttt{im2bw} which converts an image to a binary version of it. Details can also be found in Section \ref{chord_experiments.skeletonization}. Figure \ref{experiments.onset.bwmorph.skel} shows the original spectrogram and its binary version as well as the skeletonization computed out of the binary version of the original spectrogram. If we take a look at the spectrograms on the bottom, we see that the skeleton consists of a lot of thin, crisscross lines. This makes it difficult to clearly determine onsets.

\begin{figure}[tb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n20.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n20_bw.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/morphology/n20_bwskel.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/morphology/n20_bwskel_color.eps}
	\caption[Skeletonization algorithm as implemented in MATLAB.]{Skeletonization of ``rock1''. \textbf{Top left:} Original spectrogram. \textbf{Top right:} Binary image of the original spectrogram generated using the built-in MATLAB function \texttt{im2bw} with an automatically chosen threshold realized by the built-in function \texttt{graythresh}. \textbf{Bottom left:} Skeleton computed with the MATLAB function \texttt{bwmorph} for skeletonization using the binary version of the original spectrogram. \textbf{Bottom right:} Same skeleton with color information of the original spectrogram at skeleton points.}
	\label{experiments.onset.bwmorph.skel}
\end{figure}

Figure \ref{experiments.onset.skeletonization} compares binary skeletonization as implemented in MATLAB with the parabolic skeletonization proposed in Section \ref{modifications.morphology}. We notice immediately that both approaches are very different in their outcome. Parabolic skeletonization keeps the overall structure intact while thinning it at the same time where the skeletonization implemented in MATLAB gives very thin lines that are not related to the original structure in any way. Like for chord recognition, the differences between the versions where $s_x = s_y = 0.1$, $s_x = 1, s_y = 0.1$ and $s_x = 0.1, s_y = 1$ are very small. Looking carefully, we can see a blurring in horizontal direction for the spectrogram on the bottom left and an enhancement of vertical structures for the spectrogram on the bottom right. It seems that less blurring is involved for $s_x = s_y = 0.1$ leading to the conclusion that this version -- at least by judging visually -- is the best out of the three parabolic skeletonization examples. Table \ref{experiments.onset.skeletonization.table} supports this impression.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/morphology/n20_bwskel_color.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/morphology/n20_skel_01_01_001.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/morphology/n20_skel_1_01_001.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/morphology/n20_skel_01_1_001.eps}
	\caption[Different skeletonization approaches on ``rock1''.]{Different skeletonization approaches on ``rock1''. \textbf{Top left:} Skeleton computed with \texttt{bwmorph} and additional color information. \textbf{Top right:} Parabolic skeletonization with $s_x = 0.1$, $s_y = 0.1$ and a threshold of $0.01$. \textbf{Bottom left:} Same, but $s_x = 1$, $s_y = 0.1$. \textbf{Bottom right:} Same, but $s_x = 0.1$, $s_y = 1$.}
	\label{experiments.onset.skeletonization}
\end{figure}

\begin{table}[tb]
	\centering
	\caption[Comparison of skeletonization algorithms on ``rock1''.]{Comparison of skeletonization algorithms on ``rock1''. The first two skeletonizations correspond to the \texttt{bwmorph} function that is built into MATLAB. The binary image for \texttt{bwmorph} was generated using \texttt{im2bw} with an automatically chosen threshold realized by \texttt{graythresh}. Last three are parabolic skeletonizations with a threshold of $0.01$ for different values for $s_x$ and $s_y$.}
	\ \\[0.3cm]
	\begin{tabular}{cccccc}
		Method & $s_x$ & $s_y$ & Precision & Recall & F-measure \\
		\hline
		Original & -- & -- & 1.000 & 0.758 & 0.862 \\
		\hline
		MATLAB & -- & -- & 0.525 & 0.339 & 0.412 \\
		MATLAB + color & -- & -- & 0.595 & 0.355 & 0.444 \\
		\hline
		Parabolic & 0.1 & 0.1 & 0.939 & 0.742 & 0.829 \\
		Parabolic & 1 & 0.1 & 0.935 & 0.694 & 0.796 \\
		Parabolic & 0.1 & 1 & 0.918 & 0.726 & 0.811
	\end{tabular}
	\label{experiments.onset.skeletonization.table}
\end{table}

%%% ONSET EXPERIMENTS: NL-MEANS %%%
\clearpage
\section{NL-Means\index{NL-means}}\label{onset_experiments.nlmeans}

As we have seen in Section \ref{chord_experiments.nlmeans}, the modifications introduced in Sections \ref{nlmeans.refinements} and \ref{modifications.nlmeans} introduce assumptions that influence the outcome of the NL-means algorithm. Please remember that we stick to the term \emph{NL-means} to refer to an algorithm incorporating improvements from \ref{nlmeans.refinements} and \ref{modifications.nlmeans} but not the modified weighting function from \eqref{nlmeans_fast_weighting_function}, whereas \emph{fast NL-means} stands for the algorithm that additionally uses this modified weighting function. %if we do not consider the modified weighting function as proposed in equation \eqref{nlmeans_fast_weighting_function} but potentially considering all other improvements proposed in \ref{nlmeans.refinements} and \ref{modifications.nlmeans} and to \emph{fast NL-means} if we additionally use the modified weighting function \eqref{nlmeans_fast_weighting_function}.% We do not consider the basic NL-means\index{NL-means} algorithm here as well (see Section \ref{chord_experiments.nlmeans}).

In the case of NL-means and chord recognition, we had to be careful choosing what we are inpainting. As a consequence, we had to make the search window and the similarity window large enough to ensure this. Please note that NL-means has a high memory consumption, especially for large window sizes. This might lead to the exhaustion of system capacities very quickly. Experiments for onset detection with the same window sizes as for chord recognition have led to a poorer outcome. Let us show the difference on two examples. NL-means with a $21 \times 21$ search window, a similarity window of size $3 \times 21$, $\lambda = 1$ and nonadaptive distance weighting as well as fast NL-means with $\eta_1 = 0.5$ and $\eta_2 = 1.5$ were used where all other parameters remained untouched. NL-means gives on Op.\ 100 No.\ 2 by Friedrich Burgm{\"u}ller $(P, R, F) = (1.000, 0.632, 0.775)$ for precision, recall and F-measure where fast NL-means gives $(P, R, F) = (1.000, 0.647, 0.786)$. For the testfile ``classic3'', NL-means ends with $(P, R ,F) = (0.438, 0.583, 0.500)$ and fast NL-means with $(P, R, F) = (0.526, 0.833, 0.645)$. Comparing those results with the corresponding ones in Tables \ref{experiments.onset.file1.table} and \ref{experiments.onset.file12.table}, respectively, we notice that a larger window size is not suitable for this task. As a consequence, we stick to a smaller search window of size $11 \times 11$ and a smaller similarity window ($11 \times 11$ or $3 \times 11$, respectively). As a side effect, the smaller window sizes save some system resources.
%we have to stick to a smaller window since MATLAB throws an `out of memory' error for a $21 \times 21$ search window and a $3 \times 7$ similarity window on the test computer with 4 GB RAM, running a 32-bit version of MATLAB. The algorithm might run with the larger window sizes on a 64-bit MATLAB and a machine with more main memory, but this fact could not be considered in the context of this thesis. On the other hand, this does not harm very much in the context of onset detection since it is not so important what we inpaint to enhance vertical structures, but \emph{where} we inpaint. Thus, also a $11 \times 11$ window is large enough.

\subsection{Improved NL-means}
We start with the comparison between NL-means\index{NL-means} with and without adaptive weighting (see Section \ref{nlmeans.adaptive_weighting}) for different parameters $\lambda$.
\paragraph*{No directional preference.} Figure \ref{experiments.onset.nodirection.nonadaptive_adaptive} shows on the left the output spectrograms of NL-means without adaptive weighting and on the right the spectrograms that are computed with an adaptively chosen distance weight $\lambda$. First of all we may see that the spectrograms where NL-means was applied without adaptively chosen $\lambda$ are a little more blurred than the spectrograms where NL-means with adaptive weighting was used. However, the changes between the left spectrogram and its corresponding version on the right are minimal. As a general trend we observe that the higher $\lambda$ is chosen, the more blurring is involved in the spectrograms leading to poorer results with the increase of $\lambda$. Table \ref{experiments.onset.nodirection.nonadaptive_adaptive.table} verifies these observations. Furthermore, the table shows that for $\lambda = 5$ and $\lambda = 10$, a larger difference between the version without and with adaptive weighting is present. Let us exemplarily study the two spectrograms for $\lambda = 5$. More precisely, we have a look at the ``vertical peak'' at approximately 7.5 seconds. Especially at the frequencies around 3500 to 4000 we may see that NL-means without an adaptive distance weight $\lambda$ blurs very much in contrast to the version with adaptive weighting that preserves the vertical structure quite well. Similar investigations can be made e.g.\ at approximately 10.5 seconds for the frequencies around 2500 to 3000.

Except for $\lambda = 1$ without adaptive weighting, the result of NL-means is not enhanced in comparison to the original data.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n14.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeans/n14_5_5_5_5_1_n.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeans/n14_5_5_5_5_1_y.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeans/n14_5_5_5_5_5_n.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeans/n14_5_5_5_5_5_y.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeans/n14_5_5_5_5_10_n.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeans/n14_5_5_5_5_10_y.eps}
	\caption[NL-means, comparison of choosing $\lambda$ adaptively or not, quadratic windows.]{Comparison of choosing $\lambda$ adaptively or not for NL-means using the testfile ``guitar2'' (number 14 in Table \ref{appendix.table_onset_files}). All spectrograms were computed using a $11 \times 11$ search window and a $11 \times 11$ similarity window. \textbf{Top:} Original spectrogram. \textbf{Left:} $\lambda$ is not chosen adaptively. \textbf{Right:} $\lambda$ is chosen adaptively. \textbf{Middle to bottom:} $\lambda = 1, 5, 10$.}
	\label{experiments.onset.nodirection.nonadaptive_adaptive}
\end{figure}

\begin{table}[t]
	\centering
	\caption[NL-means, comparison of choosing $\lambda$ adaptively or not, quadratic windows.]{Comparison of choosing $\lambda$ adaptively or not for NL-means. A $11 \times 11$ search window and a $11 \times 11$ similarity window is used. This table corresponds to Figure \ref{experiments.onset.nodirection.nonadaptive_adaptive}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		$\lambda$ & Adaptive & Precision & Recall & F-measure \\
		\hline
		Original & -- & 0.917 & 0.805 & 0.857 \\
		\hline
		1 & no & 0.971 & 0.829 & 0.895 \\
		1 & yes & 0.917 & 0.805 & 0.857 \\
		\hline
		5 & no & 1.042 & 0.610 & 0.769 \\
		5 & yes & 1.037 & 0.683 & 0.824 \\
		\hline
		10 & no & 0.520 & 0.317 & 0.394 \\
		10 & yes & 0.591 & 0.317 & 0.413
	\end{tabular}
	\label{experiments.onset.nodirection.nonadaptive_adaptive.table}
\end{table}

\paragraph*{Vertical directional preference.} Let us proceed by exploring whether it helps to use a rectangular similarity window instead of a quadratic one. In Figure \ref{experiments.onset.direction.nonadaptive_adaptive} we have again the spectrograms of ``guitar2'', but we use now a similarity window of size $3 \times 11$ instead of $11 \times 11$. On the left side, the spectrograms for ascending $\lambda$ and nonadaptive weighting are shown from top to bottom where on the right side the spectrograms for the same values of $\lambda$ but with adaptive weighting are depicted. Visually, hardly any differences can be spotted comparing the spectrograms on the left and on the right, but the F-measure values in Table \ref{experiments.onset.direction.nonadaptive_adaptive.table} give a significant change between nonadaptive and adaptive weighting for $\lambda = 5$ and $\lambda = 10$.% Similar observations like in the case of a $11 \times 11$ similarity window instead of a $3 \times 11$ window can be made.

Please note that in Section \ref{chord_experiments.nlmeans} we have given a reason that a rectangular search window is not well suited which is also true for this task. Of course, it might be that the usage of a rectangular search window improves the outcome of the algorithm, but this highly depends on the testfile that is considered.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n14.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeans/n14_5_5_1_5_1_n.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeans/n14_5_5_1_5_1_y.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeans/n14_5_5_1_5_5_n.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeans/n14_5_5_1_5_5_y.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeans/n14_5_5_1_5_10_n.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeans/n14_5_5_1_5_10_y.eps}
	\caption[NL-means, comparison of choosing $\lambda$ adaptively or not, rectangular window.]{Same as in Figure \ref{experiments.onset.nodirection.nonadaptive_adaptive}, but with a $11 \times 11$ search window and a $3 \times 11$ similarity window.}
	\label{experiments.onset.direction.nonadaptive_adaptive}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[NL-means, comparison of choosing $\lambda$ adaptively or not, rectangular window.]{Comparison of choosing $\lambda$ adaptively or not for NL-means. A $11 \times 11$ search window and a $3 \times 11$ similarity window is used. This table corresponds to Figure \ref{experiments.onset.direction.nonadaptive_adaptive}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		$\lambda$ & Adaptive & Precision & Recall & F-measure \\
		\hline
		Original & -- & 0.917 & 0.805 & 0.857 \\
		\hline
		1 & no & 1.032 & 0.780 & 0.889 \\
		1 & yes & 0.970 & 0.780 & 0.865 \\
		\hline
		5 & no & 0.963 & 0.634 & 0.765 \\
		5 & yes & 1.036 & 0.707 & 0.841 \\
		\hline
		10 & no & 0.588 & 0.488 & 0.533 \\
		10 & yes & 0.767 & 0.561 & 0.648
	\end{tabular}
	\label{experiments.onset.direction.nonadaptive_adaptive.table}
\end{table}

\clearpage
\subsection{Fast NL-means}
We now turn our attention to the fast NL-means algorithm that uses the modified weighting function as given in equation \eqref{nlmeans_fast_weighting_function}. In this context, we compare again the results for fast NL-means with and without directional preference for the similarity window. In addition to that, we use two different parameter sets for $\eta_1$ and $\eta_2$ where the first one is $\eta_1 = 0.9, \eta_2 = 1.1$ and the second one is $\eta_1 = 0.5, \eta_2 = 1.5$.
\paragraph*{No directional preference.} Let us start with a search window of size $11 \times 11$ and an equally sized similarity window. Figure \ref{experiments.onset.nodirection.nonadaptive_adaptive.fast.etaset1} compares the impact of using the adaptive weighting function for different values of $\lambda$. Furthermore, $\eta_1 = 0.9$ and $\eta_2 = 1.1$ were used for this example. The results of the same setting but with $\eta_1 = 0.5$ and $\eta_2 = 1.5$ can be found in Figure \ref{experiments.onset.nodirection.nonadaptive_adaptive.fast.etaset2}. Since visually these figures do not differ significantly, let us focus on the results given by the quality measure. These results can be found in Table \ref{experiments.onset.nodirection.nonadaptive_adaptive.fast.etaset1.table} and Table \ref{experiments.onset.nodirection.nonadaptive_adaptive.fast.etaset2.table}, respectively.

First of all, we note that the outcome of the fast NL-means algorithm without and with adaptive weighting slightly differs. Furthermore, both parameter sets for $\eta_1$ and $\eta_2$ give the same trend concerning the increase of $\lambda$, i.e.\ the higher $\lambda$ is chosen the worse is the result. This fact can be seen on the bottom row of Figures \ref{experiments.onset.nodirection.nonadaptive_adaptive.fast.etaset1} and \ref{experiments.onset.nodirection.nonadaptive_adaptive.fast.etaset2} where the spectrograms are blurred very much. Another regularity seems to be that the version with adaptive weighting is superior in comparison to the version without adaptive weighting for higher $\lambda$.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n14.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_5_5_1_n_09_11.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_5_5_1_y_09_11.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_5_5_5_n_09_11.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_5_5_5_y_09_11.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_5_5_10_n_09_11.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_5_5_10_y_09_11.eps}
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, quadratic windows, $\eta_1 = 0.9$, $\eta_2 = 1.1$.]{Comparison of choosing $\lambda$ adaptively or not for fast NL-means using the testfile ``guitar2'' (number 14 in Table \ref{appendix.table_onset_files}). All spectrograms were computed using a $11 \times 11$ search window, a $11 \times 11$ similarity window, $\eta_1 = 0.9$ and $\eta_2 = 1.1$. \textbf{Top:} Original spectrogram. \textbf{Left:} $\lambda$ is not chosen adaptively. \textbf{Right:} $\lambda$ is chosen adaptively. \textbf{Middle to bottom:} $\lambda = 1, 5, 10$.}
	\label{experiments.onset.nodirection.nonadaptive_adaptive.fast.etaset1}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, quadratic windows, $\eta_1 = 0.9$, $\eta_2 = 1.1$.]{Comparison of choosing $\lambda$ adaptively or not for fast NL-means with a $11 \times 11$ search window, a $11 \times 11$ similarity window, $\eta_1 = 0.9$ and $\eta_2 = 1.1$. This table corresponds to Figure \ref{experiments.onset.nodirection.nonadaptive_adaptive.fast.etaset1}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		$\lambda$ & Adaptive & Precision & Recall & F-measure \\
		\hline
		Original & -- & 0.917 & 0.805 & 0.857 \\
		\hline
		1 & no & 0.943 & 0.805 & 0.868 \\
		1 & yes & 0.919 & 0.829 & 0.872 \\
		\hline
		5 & no & 1.000 & 0.659 & 0.794 \\
		5 & yes & 0.966 & 0.683 & 0.800 \\
		\hline
		10 & no & 0.625 & 0.488 & 0.548 \\
		10 & yes & 0.767 & 0.561 & 0.648
	\end{tabular}
	\label{experiments.onset.nodirection.nonadaptive_adaptive.fast.etaset1.table}
\end{table}

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n14.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_5_5_1_n_05_15.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_5_5_1_y_05_15.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_5_5_5_n_05_15.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_5_5_5_y_05_15.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_5_5_10_n_05_15.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_5_5_10_y_05_15.eps}
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, quadratic windows, $\eta_1 = 0.5$, $\eta_2 = 1.5$.]{Same as in Figure \ref{experiments.onset.nodirection.nonadaptive_adaptive.fast.etaset1}, but with $\eta_1 = 0.5$ and $\eta_2 = 1.5$.}
	\label{experiments.onset.nodirection.nonadaptive_adaptive.fast.etaset2}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, quadratic windows, $\eta_1 = 0.5$, $\eta_2 = 1.5$.]{Comparison of choosing $\lambda$ adaptively or not for fast NL-means. A $11 \times 11$ search window, a $11 \times 11$ similarity window and $\eta_1 = 0.5$, $\eta_2 = 1.5$ is used. This table corresponds to Figure \ref{experiments.onset.nodirection.nonadaptive_adaptive.fast.etaset2}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		$\lambda$ & Adaptive & Precision & Recall & F-measure \\
		\hline
		Original & -- & 0.917 & 0.805 & 0.857 \\
		\hline
		1 & no & 0.944 & 0.829 & 0.883 \\
		1 & yes & 0.919 & 0.829 & 0.872 \\
		\hline
		5 & no & 1.000 & 0.659 & 0.794 \\
		5 & yes & 1.000 & 0.683 & 0.812 \\
		\hline
		10 & no & 0.429 & 0.293 & 0.348 \\
		10 & yes & 0.483 & 0.341 & 0.400
	\end{tabular}
	\label{experiments.onset.nodirection.nonadaptive_adaptive.fast.etaset2.table}
\end{table}

\paragraph*{Vertical directional preference.} We now turn to the case that the similarity window has a rectangular shape, i.e.\ $3 \times 11$.

Figure \ref{experiments.onset.direction.nonadaptive_adaptive.fast.etaset1} contains the spectrograms computed by the application of the fast NL-means algorithm using $\eta_1 = 0.9$ and $\eta_2 = 1.1$. Figure \ref{experiments.onset.direction.nonadaptive_adaptive.fast.etaset2} gives the results for fast NL-means with $\eta_1 = 0.5$ and $\eta_2 = 1.5$. The first thing we notice in both figures is that hardly any blurring is involved in the result. Nevertheless, with a higher $\lambda$ also the blurring increases.% The values of precision, recall and F-measure given in Tables \ref{experiments.onset.direction.nonadaptive_adaptive.fast.etaset1.table} and \ref{experiments.onset.direction.nonadaptive_adaptive.fast.etaset2.table} also verify that even larger $\lambda$ give enhancements.%for high values of $\lambda$ the result is good. 

%Depending on the testfile and on the choice of $\eta_1$ and $\eta_2$
%For $\eta_1 = 0.9$ and $\eta_2 = 1.1$, the result with the largest F-measure can be achieved for $\lambda = 5$ on this testfile (see Table \ref{experiments.onset.direction.nonadaptive_adaptive.fast.etaset1.table}) where $\lambda = 1$ gives in general the best results (e.g.\ Table \ref{experiments.onset.direction.nonadaptive_adaptive.fast.etaset2.table}).
Let us assume that $\eta_1 = 0.9$ and $\eta_2 = 1.1$ are chosen. In Table \ref{experiments.onset.direction.nonadaptive_adaptive.fast.etaset1.table}, we see that $\lambda = 5$ without adaptive weighting gives the highest F-measure. This outcome is not expected, since the computation with $\lambda = 1$ results in less blurring in the spectrograms compared to the other values of $\lambda$. The fact that the computation with $\lambda = 5$ gives together with a higher precision also a lower recall compared to the original data supports our intuition that this result should be taken with care. In contrast, for $\lambda = 1$ we have an increase in precision together with the same recall as for the original data. If $\eta_1 = 0.5$ and $\eta_2 = 1.5$ are used, the result for $\lambda = 1$ without adaptive weighting is superior to all other results, compare Table \ref{experiments.onset.direction.nonadaptive_adaptive.fast.etaset2.table}.

But why is there hardly any blurring involved in this setting? This circumstance results from the usage of the rectangular similarity window which limits the amount of inpainting in horizontal direction and thus reduces blurring.

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n14.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_1_5_1_n_09_11.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_1_5_1_y_09_11.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_1_5_5_n_09_11.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_1_5_5_y_09_11.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_1_5_10_n_09_11.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_1_5_10_y_09_11.eps}
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, rectangular window, $\eta_1 = 0.9$, $\eta_2 = 1.1$.]{Comparison of choosing $\lambda$ adaptively or not for fast NL-means using the testfile ``guitar2'' (number 14 in Table \ref{appendix.table_chord_files}). All spectrograms were computed using a $11 \times 11$ search window, a $3 \times 11$ similarity window, $\eta_1 = 0.9$ and $\eta_2 = 1.1$. \textbf{Top:} Original spectrogram. \textbf{Left:} $\lambda$ is not chosen adaptively. \textbf{Right:} $\lambda$ is chosen adaptively. \textbf{Middle to bottom:} $\lambda = 1, 5, 10$.}
	\label{experiments.onset.direction.nonadaptive_adaptive.fast.etaset1}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, rectangular window, $\eta_1 = 0.9$, $\eta_2 = 1.1$.]{Comparison of choosing $\lambda$ adaptively or not for fast NL-means with $\eta_1 = 0.9$ and $\eta_2 = 1.1$ where a $11 \times 11$ search window and a $3 \times 11$ similarity window is used. This table corresponds to Figure \ref{experiments.onset.direction.nonadaptive_adaptive.fast.etaset1}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		$\lambda$ & Adaptive & Precision & Recall & F-measure \\
		\hline
		Original & -- & 0.917 & 0.805 & 0.857 \\
		\hline
		1 & no & 0.971 & 0.805 & 0.880 \\
		1 & yes & 0.943 & 0.805 & 0.868 \\
		\hline
		5 & no & 1.032 & 0.780 & 0.889 \\
		5 & yes & 1.000 & 0.780 & 0.877 \\
		\hline
		10 & no & 1.000 & 0.780 & 0.877 \\
		10 & yes & 1.000 & 0.780 & 0.877
	\end{tabular}
	\label{experiments.onset.direction.nonadaptive_adaptive.fast.etaset1.table}
\end{table}

\begin{figure}[htb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n14.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_1_5_1_n_05_15.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_1_5_1_y_05_15.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_1_5_5_n_05_15.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_1_5_5_y_05_15.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_1_5_10_n_05_15.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n14_5_5_1_5_10_y_05_15.eps}
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, rectangular window, $\eta_1 = 0.5$, $\eta_2 = 1.5$.]{Same as in Figure \ref{experiments.onset.direction.nonadaptive_adaptive.fast.etaset1}, but with $\eta_1 = 0.5$ and $\eta_2 = 1.5$.}
	\label{experiments.onset.direction.nonadaptive_adaptive.fast.etaset2}
\end{figure}

\begin{table}[ht]
	\centering
	\caption[Fast NL-means, $\lambda$ (non-)adaptive, rectangular window, $\eta_1 = 0.5$, $\eta_2 = 1.5$.]{Comparison of choosing $\lambda$ adaptively or not for fast NL-means with $\eta_1 = 0.5$ and $\eta_2 = 1.5$. Furthermore, a $11 \times 11$ search window and a $3 \times 11$ similarity window is used. This table corresponds to Figure \ref{experiments.onset.direction.nonadaptive_adaptive.fast.etaset2}.}
	\ \\[0.3cm]
	\begin{tabular}{ccccc}
		$\lambda$ & Adaptive & Precision & Recall & F-measure \\
		\hline
		Original & -- & 0.917 & 0.805 & 0.857 \\
		\hline
		1 & no & 1.000 & 0.805 & 0.892 \\
		1 & yes & 0.971 & 0.805 & 0.880 \\
		\hline
		5 & no & 0.969 & 0.756 & 0.849 \\
		5 & yes & 1.033 & 0.756 & 0.873 \\
		\hline
		10 & no & 0.889 & 0.585 & 0.706 \\
		10 & yes & 0.929 & 0.634 & 0.754
	\end{tabular}
	\label{experiments.onset.direction.nonadaptive_adaptive.fast.etaset2.table}
\end{table}

%%% ONSET EXPERIMENTS: DISCUSSION/SUMMARY %%%
\clearpage
\section{Discussion and Summary}\label{onset_experiments.summary}
This section concludes the experiments for the onset detection problem. We do so by using a standard parameter set for EED, CED, skeletonization, NL-means and fast NL-means on all the testfiles and compare for each file how well the algorithms perform. Like in Section \ref{chord_experiments.summary}, we are interested in a reasonable starting point concerning the choice of parameters for the algorithms under investigation, since in real-world examples, usually no ground truth is available. As it is for the chord recognition problem, standardized parameter sets are crucial for an automated application of the methods. Since the impact of the algorithms is visually demonstrated in the previous sections, we omit to give the spectrograms like we did in Section \ref{chord_experiments.summary}.

For the computations, the following parameter sets are applied and preference for vertical structures is always used except for skeletonization.
\begin{itemize}
	\item EED: \\
		$\lambda = 1$, $\sigma = 0.5$, $\tau = 0.25$, $t = 20$.
	\item CED: \\
		$C = 1$, $\sigma = 0.5$, $\rho = 5$, $\alpha = 0.01$, $\tau = 0.25$, $t = 200$.
	\item Skeletonization: \\
		$s_x = 0.1$, $s_y = 0.1$, threshold = $0.01$.
	\item NL-means: \\
		$11 \times 11$ search window, $3 \times 11$ similarity window, $\lambda = 1$, no adaptive weighting.
	\item Fast NL-means: \\
		$11 \times 11$ search window, $3 \times 11$ similarity window, $\eta_1 = 0.5$, $\eta_2 = 1.5$, $\lambda = 1$, no adaptive weighting.
\end{itemize}

\paragraph*{Performance comparison across all approaches.} Concerning diffusion, we should keep in mind the following. The spectrograms of the testfiles for onset detection investigated in this thesis are in most cases very dense. Thus, we have to be very careful when choosing the diffusion time; choosing it too large might lead to unwanted results.

The outcome of the processing by the algorithms investigated in this thesis is given for each testfile in Tables \ref{experiments.onset.file1.table} to \ref{experiments.onset.file23.table}. As a general trend, we note that the class of diffusion processes beats the class of NL-means algorithms on the testfiles, compare e.g.\ Tables \ref{experiments.onset.file6.table} to \ref{experiments.onset.file9.table}. Also, skeletonization outperforms both NL-means approaches on several testfiles, e.g.\ on the testfile ``jazz3'' (see Table \ref{experiments.onset.file17.table}). Please note that for ``jazz3'', skeletonization also beats diffusion. Furthermore, the NL-means algorithm that does not use the weighting function as given in equation \eqref{nlmeans_fast_weighting_function} gives better results in comparison to the fast NL-means. Interestingly, NL-means and fast NL-means beat diffusion on testfiles where mainly vocals are present, compare Tables \ref{experiments.onset.file4.table} and \ref{experiments.onset.file5.table}, respectively. In contrast, diffusion gives a slightly higher F-measure for ``classic2'' even though this file also contains vocals, compare Table \ref{experiments.onset.file11.table}. The reason might be that the singing starts at approximately half of the audio excerpt where the first half contains only piano. In addition, the low frequencies on the bottom of the spectrogram are already blurred throughout the unprocessed testfile, compare also the corresponding original spectrogram in Figure \ref{appendix.figure_onset_files2}.

So far, we compared the algorithms among each other. Considering the overall improvement, i.e.\ whether it is beneficial to apply one of the algorithms on the testfiles, we notice that for some examples no improvement according to precision, recall and F-measure is achieved at all, see Tables \ref{experiments.onset.file3.table}, \ref{experiments.onset.file7.table}, \ref{experiments.onset.file8.table}, \ref{experiments.onset.file11.table}, \ref{experiments.onset.file12.table}, \ref{experiments.onset.file16.table}, \ref{experiments.onset.file18.table} and \ref{experiments.onset.file20.table}.

\paragraph*{Runtime considerations.} Even if runtime is not in the main focus of this thesis, we don't want to leave it out of consideration. Table \ref{experiments.onset.file9.runtime.table} gives the runtime of the algorithms investigated in this thesis on the testfile ``cello1'' (number 9 in Table \ref{appendix.table_onset_files}) using the respective parameter sets as they are given before. In general, the runtime of the algorithms is lower than for the chord recognition problem, compare Table \ref{experiments.chord.file5.runtime.table}, because we are using less iterations for diffusion or use smaller window sizes in the NL-means algorithms.

\paragraph*{Combination of different filters.} A natural idea (like in the case of chord recognition) is to combine two of the methods to further enhance the spectrograms. Since diffusion and NL-means surpass skeletonization on the problems, we combine EED and fast NL-means. Applied on its own, edge-enhancing diffusion needs in general few iteration steps to enhance a spectrogram for the task of onset detection. In combination with fast NL-means, the small changes of the diffusion process are somehow compensated by the non-local inpainting which gives a result very similar to the one for fast NL-means alone. Thus, we modify the parameter set proposed earlier in this section by increasing the number of iterations for the EED algorithm to $t = 200$ while leaving the rest of the EED parameter set and the parameters for fast NL-means untouched. Table \ref{experiments.onset.fast_nlmeans.preprocessing_postprocessing.table} shows the results of fast NL-means as preprocessing or postprocessing operation for edge-enhancing diffusion. Figure \ref{experiments.onset.fast_nlmeans.preprocessing_postprocessing} contains the corresponding spectrograms. For the sake of readability, we use \emph{FNLM} instead of fast NL-means in Table \ref{experiments.onset.fast_nlmeans.preprocessing_postprocessing.table}.

The first thing we notice when looking at the spectrograms in Figure \ref{experiments.onset.fast_nlmeans.preprocessing_postprocessing} is that the application of EED, be it either as preprocessing, postprocessing or alone, introduces some amount of blurring. Secondly, the spectrograms on the middle left and on the bottom look quite similar. Thus, it is difficult to say by visual comparison why EED is superior to the versions where fast NL-means is used once as preprocessing and once as postprocessing step. Except for pure EED, the results are quite close based on the values for precision, recall and F-measure given in Table \ref{experiments.onset.fast_nlmeans.preprocessing_postprocessing.table}.% Furthermore, all efforts in this experiment to make the overall quality better are not successful since the original, unprocessed data still has better precision, recall and F-measure.

\paragraph*{Final remark.} Onset detection seems to be insensitive to noise that appears locally which is the type of noise we try to reduce by image processing methods. Thus, we are only able to slightly enhance the spectrograms.
%where big improvements might need more efforts in adapting image processing algorithms by the use of knowledge from audio processing.

\clearpage
\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 1.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for Op.\ 100 No.\ 2 by Friedrich Burgm{\"u}ller (file number 1 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.979 & 0.676 & 0.800 \\
		\hline
		EED & 1.000 & 0.691 & 0.817 \\
		CED & 0.979 & 0.676 & 0.800 \\
		\hline
		Skeletonization & 0.979 & 0.676 & 0.800 \\
		\hline
		NL-means & 1.000 & 0.662 & 0.796 \\
		Fast NL-means & 1.000 & 0.662 & 0.796
	\end{tabular}
	\label{experiments.onset.file1.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 2.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for Beethoven's Fifth in a Bernstein interpretation (file number 2 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.552 & 0.296 & 0.386 \\
		\hline
		EED & 0.538 & 0.389 & 0.452 \\
		CED & 0.500 & 0.352 & 0.413 \\
		\hline
		Skeletonization & 0.429 & 0.278 & 0.337 \\
		\hline
		NL-means & 0.565 & 0.241 & 0.338 \\
		Fast NL-means & 0.524 & 0.204 & 0.293
	\end{tabular}
	\label{experiments.onset.file2.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 3.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for Hungarian Dance by Brahms (file number 3 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.983 & 0.731 & 0.838 \\
		\hline
		EED & 0.950 & 0.731 & 0.826 \\
		CED & 0.966 & 0.731 & 0.832 \\
		\hline
		Skeletonization & 0.966 & 0.731 & 0.832 \\
		\hline
		NL-means & 0.964 & 0.692 & 0.806 \\
		Fast NL-means & 0.964 & 0.692 & 0.806
	\end{tabular}
	\label{experiments.onset.file3.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 4.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``Happy Birthday'' (file number 4 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.383 & 0.692 & 0.493 \\
		\hline
		EED & 0.455 & 0.769 & 0.571 \\
		CED & 0.426 & 0.769 & 0.548 \\
		\hline
		Skeletonization & 0.306 & 0.577 & 0.400 \\
		\hline
		NL-means & 0.556 & 0.769 & 0.645 \\
		Fast NL-means & 0.556 & 0.769 & 0.645
	\end{tabular}
	\label{experiments.onset.file4.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 5.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``In Frankrijk buiten de poorten'' (file number 5 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.500 & 0.737 & 0.596 \\
		\hline
		EED & 0.517 & 0.789 & 0.625 \\
		CED & 0.500 & 0.737 & 0.596 \\
		\hline
		Skeletonization & 0.483 & 0.737 & 0.583 \\
		\hline
		NL-means & 0.583 & 0.737 & 0.651 \\
		Fast NL-means & 0.560 & 0.737 & 0.636
	\end{tabular}
	\label{experiments.onset.file5.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 6.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for Beethoven's Fifth as MIDI file (file number 6 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 1.133 & 0.405 & 0.596 \\
		\hline
		EED & 1.161 & 0.429 & 0.626 \\
		CED & 1.097 & 0.405 & 0.591 \\
		\hline
		Skeletonization & 0.903 & 0.333 & 0.487 \\
		\hline
		NL-means & 1.040 & 0.310 & 0.477 \\
		Fast NL-means & 1.045 & 0.274 & 0.434
	\end{tabular}
	\label{experiments.onset.file6.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 7.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for String Quartet No.\ 2 by Alexander Borodin (file number 7 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.396 & 0.462 & 0.426 \\
		\hline
		EED & 0.346 & 0.462 & 0.396 \\
		CED & 0.389 & 0.449 & 0.417 \\
		\hline
		Skeletonization & 0.372 & 0.449 & 0.407 \\
		\hline
		NL-means & 0.372 & 0.372 & 0.372 \\
		Fast NL-means & 0.373 & 0.359 & 0.366
	\end{tabular}
	\label{experiments.onset.file7.table}
\end{table}

%\clearpage
\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 8.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for Waltz No.\ 2 from Jazz Suite No.\ 2 by Dmitri Shostakovich (file number 8 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.971 & 0.878 & 0.922 \\
		\hline
		EED & 0.980 & 0.861 & 0.917 \\
		CED & 0.970 & 0.852 & 0.907 \\
		\hline
		Skeletonization & 0.939 & 0.800 & 0.864 \\
		\hline
		NL-means & 0.961 & 0.852 & 0.903 \\
		Fast NL-means & 0.970 & 0.835 & 0.897
	\end{tabular}
	\label{experiments.onset.file8.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 9.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``cello1'' (file number 9 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.932 & 0.846 & 0.887 \\
		\hline
		EED & 0.984 & 0.938 & 0.961 \\
		CED & 0.948 & 0.846 & 0.894 \\
		\hline
		Skeletonization & 0.930 & 0.815 & 0.869 \\
		\hline
		NL-means & 0.981 & 0.785 & 0.872 \\
		Fast NL-means & 0.962 & 0.769 & 0.855
	\end{tabular}
	\label{experiments.onset.file9.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 10.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``clarinet1'' (file number 10 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.170 & 0.211 & 0.188 \\
		\hline
		EED & 0.196 & 0.237 & 0.214 \\
		CED & 0.174 & 0.211 & 0.190 \\
		\hline
		Skeletonization & 0.143 & 0.184 & 0.161 \\
		\hline
		NL-means & 0.205 & 0.237 & 0.220 \\
		Fast NL-means & 0.196 & 0.237 & 0.214
	\end{tabular}
	\label{experiments.onset.file10.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 11.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``classic2'' (file number 11 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.625 & 0.714 & 0.667 \\
		\hline
		EED & 0.623 & 0.673 & 0.647 \\
		CED & 0.596 & 0.694 & 0.642 \\
		\hline
		Skeletonization & 0.630 & 0.694 & 0.660 \\
		\hline
		NL-means & 0.646 & 0.633 & 0.639 \\
		Fast NL-means & 0.644 & 0.592 & 0.617
	\end{tabular}
	\label{experiments.onset.file11.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 12.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``classic3'' (file number 12 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.588 & 0.833 & 0.690 \\
		\hline
		EED & 0.290 & 0.750 & 0.419 \\
		CED & 0.435 & 0.833 & 0.571 \\
		\hline
		Skeletonization & 0.462 & 0.500 & 0.480 \\
		\hline
		NL-means & 0.556 & 0.833 & 0.667 \\
		Fast NL-means & 0.556 & 0.833 & 0.667
	\end{tabular}
	\label{experiments.onset.file12.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 13.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``distguit1'' (file number 13 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.833 & 0.750 & 0.789 \\
		\hline
		EED & 0.833 & 0.750 & 0.789 \\
		CED & 0.842 & 0.800 & 0.821 \\
		\hline
		Skeletonization & 0.833 & 0.750 & 0.789 \\
		\hline
		NL-means & 0.824 & 0.700 & 0.757 \\
		Fast NL-means & 0.824 & 0.700 & 0.757
	\end{tabular}
	\label{experiments.onset.file13.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 14.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``guitar2'' (file number 14 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.917 & 0.805 & 0.857 \\
		\hline
		EED & 0.970 & 0.780 & 0.865 \\
		CED & 0.919 & 0.829 & 0.872 \\
		\hline
		Skeletonization & 0.882 & 0.732 & 0.800 \\
		\hline
		NL-means & 1.032 & 0.780 & 0.889 \\
		Fast NL-means & 1.000 & 0.805 & 0.892
	\end{tabular}
	\label{experiments.onset.file14.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 15.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``guitar3'' (file number 15 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 1.000 & 0.828 & 0.906 \\
		\hline
		EED & 1.000 & 0.810 & 0.895 \\
		CED & 0.980 & 0.845 & 0.907 \\
		\hline
		Skeletonization & 1.000 & 0.793 & 0.885 \\
		\hline
		NL-means & 1.000 & 0.879 & 0.936 \\
		Fast NL-means & 1.000 & 0.879 & 0.936
	\end{tabular}
	\label{experiments.onset.file15.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 16.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``jazz2'' (file number 16 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.900 & 0.482 & 0.628 \\
		\hline
		EED & 0.897 & 0.464 & 0.612 \\
		CED & 0.900 & 0.482 & 0.628 \\
		\hline
		Skeletonization & 0.862 & 0.446 & 0.588 \\
		\hline
		NL-means & 0.923 & 0.429 & 0.585 \\
		Fast NL-means & 0.889 & 0.429 & 0.578
	\end{tabular}
	\label{experiments.onset.file16.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 17.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``jazz3'' (file number 17 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 1.026 & 0.639 & 0.788 \\
		\hline
		EED & 1.027 & 0.623 & 0.776 \\
		CED & 1.026 & 0.639 & 0.788 \\
		\hline
		Skeletonization & 1.000 & 0.672 & 0.804 \\
		\hline
		NL-means & 0.955 & 0.344 & 0.506 \\
		Fast NL-means & 0.963 & 0.426 & 0.591
	\end{tabular}
	\label{experiments.onset.file17.table}
\end{table}

\clearpage
\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 18.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``piano1'' (file number 18 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 1.000 & 1.000 & 1.000 \\
		\hline
		EED & 1.000 & 1.000 & 1.000 \\
		CED & 1.000 & 1.000 & 1.000 \\
		\hline
		Skeletonization & 1.000 & 0.950 & 0.974 \\
		\hline
		NL-means & 1.000 & 1.000 & 1.000 \\
		Fast NL-means & 1.000 & 0.950 & 0.974
	\end{tabular}
	\label{experiments.onset.file18.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 19.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``pop1'' (file number 19 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.771 & 0.844 & 0.806 \\
		\hline
		EED & 0.818 & 0.844 & 0.831 \\
		CED & 0.794 & 0.844 & 0.818 \\
		\hline
		Skeletonization & 0.688 & 0.688 & 0.687 \\
		\hline
		NL-means & 0.794 & 0.844 & 0.818 \\
		Fast NL-means & 0.800 & 0.875 & 0.836
	\end{tabular}
	\label{experiments.onset.file19.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 20.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``rock1'' (file number 20 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 1.000 & 0.758 & 0.862 \\
		\hline
		EED & 1.000 & 0.726 & 0.841 \\
		CED & 1.000 & 0.758 & 0.862 \\
		\hline
		Skeletonization & 0.939 & 0.742 & 0.829 \\
		\hline
		NL-means & 1.000 & 0.710 & 0.830 \\
		Fast NL-means & 1.000 & 0.710 & 0.830
	\end{tabular}
	\label{experiments.onset.file20.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 21.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``sax1'' (file number 21 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.500 & 0.800 & 0.615 \\
		\hline
		EED & 0.500 & 0.900 & 0.643 \\
		CED & 0.471 & 0.800 & 0.593 \\
		\hline
		Skeletonization & 0.500 & 0.800 & 0.615 \\
		\hline
		NL-means & 0.500 & 0.800 & 0.615 \\
		Fast NL-means & 0.588 & 1.000 & 0.741
	\end{tabular}
	\label{experiments.onset.file21.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 22.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``trumpet1'' (file number 22 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.976 & 0.683 & 0.804 \\
		\hline
		EED & 0.953 & 0.683 & 0.796 \\
		CED & 0.956 & 0.717 & 0.819 \\
		\hline
		Skeletonization & 0.909 & 0.667 & 0.769 \\
		\hline
		NL-means & 0.976 & 0.683 & 0.804 \\
		Fast NL-means & 0.953 & 0.683 & 0.796
	\end{tabular}
	\label{experiments.onset.file22.table}
\end{table}

\begin{table}[ht]
	\centering
	\caption[Comparison of different methods for onset detection testfile number 23.]{Comparison of EED, CED, skeletonization, NL-means and fast NL-means for ``violin2'' (file number 23 in Table \ref{appendix.table_onset_files}).}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Method & Precision & Recall & F-measure \\
		\hline
		Original & 0.909 & 0.506 & 0.650 \\
		\hline
		EED & 0.933 & 0.532 & 0.677 \\
		CED & 0.932 & 0.519 & 0.667 \\
		\hline
		Skeletonization & 0.905 & 0.481 & 0.628 \\
		\hline
		NL-means & 1.000 & 0.481 & 0.650 \\
		Fast NL-means & 1.000 & 0.481 & 0.650
	\end{tabular}
	\label{experiments.onset.file23.table}
\end{table}

\begin{table}[tb]
	\centering
	\caption{Comparison of algorithm runtime for ``cello1''.}
	\ \\[0.3cm]
	\begin{tabular}{cr}
		Method & Runtime [s] \\
		\hline
		EED & 0.693905 \\
		CED & 19.238360 \\
		\hline
		Skeletonization & 104.406383 \\
		\hline
		NL-means & 229.284294 \\
		Fast NL-means & 151.289294
	\end{tabular}
	\label{experiments.onset.file9.runtime.table}
\end{table}

\begin{table}[tb]
	\centering
	\caption[Fast NL-means as pre-/postprocessing step for diffusion.]{Fast NL-means as preprocessing or postprocessing step for diffusion. This table gives the corresponding values for precision, recall and F-measure for the spectrograms given in Figure \ref{experiments.onset.fast_nlmeans.preprocessing_postprocessing}.}
	\ \\[0.3cm]
	\begin{tabular}{cccc}
		Application chain & Precision & Recall & F-measure \\
		\hline
		Original & 1.026 & 0.639 & 0.788 \\
		\hline
		EED & 1.028 & 0.607 & 0.763 \\
		FNLM & 0.963 & 0.426 & 0.591 \\
		\hline
		EED $\rightarrow$ FNLM & 0.964 & 0.443 & 0.607 \\
		FNLM $\rightarrow$ EED & 0.962 & 0.410 & 0.575
	\end{tabular}
	\label{experiments.onset.fast_nlmeans.preprocessing_postprocessing.table}
\end{table}

\begin{figure}[tb]
	\centering
	\includegraphics[scale=0.42]{figures/experiments/onset/original/n17.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/eed/n17_200_1_0.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/nlmeansfast/n17_5_5_1_5_1_n_05_15.eps} \\
	\includegraphics[scale=0.42]{figures/experiments/onset/chains/n17_eed_fnlm.eps}
	\includegraphics[scale=0.42]{figures/experiments/onset/chains/n17_fnlm_eed.eps}	
	\caption[Fast NL-means as pre-/postprocessing step for diffusion.]{Fast NL-means as preprocessing or postprocessing step for diffusion. \textbf{Top:} Original spectrogram of ``jazz3'' (number 17 in Table \ref{appendix.table_onset_files}). \textbf{Middle left:} Spectrogram after applying EED with 200 iterations. \textbf{Middle right:} Fast NL-means with a $11 \times 11$ search window and a $3 \times 11$ similarity window, $\eta_1 = 0.5$, $\eta_2 = 1.5$, $\lambda = 1$ and no adaptive weighting. \textbf{Bottom left:} Application of a diffusion process followed by fast NL-means. \textbf{Bottom right:} Application of fast NL-means followed by a diffusion process.}
	\label{experiments.onset.fast_nlmeans.preprocessing_postprocessing}
\end{figure}
