A Study on Broadcast Networks for Music Genre Classification

Fig. 1 shows the proposed architecture of the broadcast network. Inspired by BBNN[22] and the way in which the ear uses distinct filters to analyze sound characteristics [9, 23], we propose broadcast networks comprising multiple feature extractors that render different scales of feature maps. And, extending the inception scheme[9], we used three convolutions with two different kernel sizes along with a max-pooling layer, and an activation layer for each inception block. Moreover, we used residual connections as they smooth the loss landscape, and allow for deeper architectures[24]. Furthermore, residual connections transmit low-level features to all layers, which are crucial to MGC[21]. As such, our module aims at exploiting multiple features in the classifier layer. Compared to the baseline broadcast-bottom up network[22], we implement five relevant variations involving: (1) the removal of the 3x3 convolution, (2) the replacement of the 5x5 convolution by a 3x3 convolution, (3) the extension of the number of blocks to four instead of three, (4) the change of the kernel initializer to lecun normal, and (5) the replacement of the asymmetric factorized 3x3 convolutions. And, for each block, we adopt filter sizes of 1x1, 1x1, 3x3, with a 1x1 convolution before the latter to reduce its channel size and create a bottleneck representation [9]. We also factorized the 3x3 convolutions to 3x1 and 1x3 convolutions as suggested by [10]. The above-mentioned implementations are motivated by the following notions:

• The removal of 3x3 convolutions and the replacement of 5x5 convolutions allowed our model to use more feature maps under the same complexity constraints ($\sim$ 180k parameters), which enriches our model capacity and its ability to capture more textural content from their corresponding spectrograms. Moreover, they provide a better generalization and more depth under the same constraints, which is shown in the training of FMA dataset, Table II.

• By adding one additional block, we compensate for the capacity loss from removing the 3x3 convolutions.

• Changing the kernel initializer was experimentally proven to increase accuracy significantly.

• Finally, the asymmetric factorized convolutions was implemented as inspired by the results from[10].

As shown in Fig. 1, the proposed network consists of $L$ connected modules with residual connections in-between, allowing higher-level layers to receive all feature maps from previous layers. We denote $X_{SL}$ rendered from shallow layers as the input of our module, $L$ is the number of blocks of our module. For simplicity and without loss of generality, we fixed $L=4$. Thus, the input of the $l$-th block, $l=1,...,L$, can be represented as:

where $[X_{SL},X_1,...,X_{l-1}]$ refers to the concatenation of the feature maps produced by blocks $0,...,l-1$, and $f_1$ is a composite function of all operations in the inception block [22].

As shown in Fig. 1, the model comprises 22 convolution layers (5 are 3x3, and 17 are 1x1) with 4 inception blocks to increase the generalizability and allow the extraction of low-level features. As recommended by [25], we used Batch Normalization after each convolution layer followed by a ReLU activation; however, we changed the order in the inception inspired by the findings of [26]. Batch Normalization does not require regularization and solves the dying ReLU problem. As such, all the layers of the network can be summarized into four different parts: the shallow feature extraction, the proposed module, the transition layers, and the decision layers. The model aims to learn the parameter $\theta$ of a function $F\left(X_0|\theta \right)$ that maps the input spectrogram $X_0$ to its corresponding genre $p$, as follows:

We used a block-based architecture to simplify the design process and have a constant growth rate to facilitate the scaling towards large datasets. Our model has a growth rate of $k_l=k_{l-1}+4\times f$, where $k_l$ is the output of the $l^{th}$-layer, and $f$ is the number of convolution filters. We fixed the number of convolution filters to be 32. Hence, the output channel dimension of the proposed module is 544, which is reduced in later steps via the transition layer for the classification step.

Table I shows the characteristics of the datasets for MGC used in our study. Basically, we used the well-known and challenging datasets in the community, as follows:

• GTZAN Dataset. Introduced by [1], and being the first publicly available and well-structured benchmark, the GTZAN dataset is a popular dataset in MGC [27] [28]. Basically, the GTZAN is a collection of 10 popular genres (blues, classical, country, disco, hip-hop, jazz, metal, pop, reggae, rock) with 100 audio files each, all having a length of 30 seconds.

• Extended Ballroom Dataset. Assembled for the ISMIR 2004 rhythm description contest, the ballroom dataset includes 698 30-second music clips grouped into eight categories reflecting distinct ballroom dances. The extended ballroom dataset, introduced by [29], has 4180 30 second tracks and covers 13 distinct genres, representing an improvement to the original ballroom dataset with about six times the number of tracks with superior audio quality.

• HOMBURG Dataset. The HOMBURG dataset, presented by [30], is a freely available benchmark dataset for audio classification and clustering. The HOMBURG dataset consists of 1886 songs taken from the Garageband site and classified into nine unbalanced genres. Each sample taken randomly from the song lasts for 10 seconds.

• Free Music Archive Dataset (FMA). Introduced by [27], this dataset is much larger and diverse compared to the datasets mentioned above. Basically, it comprises four subsets: Full, Large, Medium, and Small. FMA is a freely available dataset used to evaluate a variety of MIR tasks. It has 917 GB and 343 days of licensed audio from 106,574 tracks from 16,341 artists and 14,854 albums, organized into 161 genres in a hierarchical taxonomy. Due to hardware constraints in environment, we used the Small set which contains about 8,000 tracks of 30s songs distributed over eight balanced genres (7.2 GiB). As such, compared to the GTZAN and the Extended Ballroom, the above-mentioned selection is comparatively larger, thus we use it to evaluate the ability to generalize and scale.

For fairness, we used the same preprocessing technique used by baseline[22]. The Mel-Spectrogram of the audio file was fed to the model as input. Spectrograms were created using a Short Time Fourier Transform (STFT), and then converted into the Mel scale by a logarithmic scale to the frequency axis of the spectrogram in the librosa toolbox [31]. 128 Mel-filters (bands) were used to represent the audible spectrum range 0-22050 Hz. We used a frame length of 2048 and a hop size of 1024. Due to the previous setup, the Mel-Spectrogram was of size 646x128.

We used a similar setups for fairness in comparisons with the baseline BBNN model. Our model was built in Keras and trained on Quadro RTX 5000. The model took half an hour to train for 100 epochs on GTZAN. We used a small batch size of 8 since the input resolution is high (647 x 128). Results were measured using a 10-fold validation for each dataset and each variation of the model, and we reported the highest test accuracy. All models were trained using ADAM due to being suitable for noisy gradients, which is the case of high-textural spectrograms[32]. Inspired by the BBNN model, we set the initial learning rate as 0.01, which was decreased by a factor of 0.5 when the loss stopped improving for three epochs. To avoid overfitting, we set up an early stopping mechanism when the training loss stopped decreasing.

To show the effectiveness of our proposed model, Fig. 2 shows the confusion matrices of our proposed model in all the datasets, Table II shows the performance comparison (validation-testing) between our proposed model and the baseline, and Fig. 3 shows the learning performance in the training, validation and testing. By observing Fig. 2, Table II and Fig. 3, we can observe the following facts:

• GTZAN Results. As shown by Fig. 3 (a), our model achieved a smaller loss. We also observed that BBNN was unable to differentiate between country, metal, and rock as they were different on the high-frequency components and similar on the low-frequencies. Since our model focuses more on localization, i.e., it allows to capture the high-frequency components, it achieved a much higher accuracy score on the three genres, as shown in Fig. 2 (a). Our model confused hip-hop and pop as the architecture discriminates low-frequency components. Our model boosted the test accuracy by +1%, as shown by Table II.

• Extended Ballroom Results. As the extended ballroom dataset has four times the samples of the GTZAN, as shown in Table I, the model must use its parameters efficiently. Although both our model and the BBNN show similar training loss values, our model exceeds the performance of the BBNN on the test set but not on the validation set, as shown in Fig. 3 (b). Nonetheless, our model achieves slightly lower validation accuracy than the BBNN model $-1.4\%$, and a higher test accuracy +0.8%, as shown in Table II. By observing at the confusion matrix in Fig. 2 (b), our model was unable to differentiate betwen Rumba, Slow Waltz, and Waltz. Also, pasodoble was confused with tango, yet we consider this phenomenon due to the low number of samples of the pasodoble genre.

• HOMBURG Results. To evaluate the generalization ability to perform in datasets different than GTZAN and Ballroom Dataset, we experimented with HOMBURG dataset, which has the equivalent size of the datasets used to test the baseline model. Our model outperformed the baseline with +3% accuracy as shown in table II. We also observed that the model was most successful at identifying jazz, rock, and hip-hop, yet the model failed at identifying the funk/soul genre as it is a hybrid genre. Furthermore, our model confused jazz and blues in some cases as they both have very similar low-frequency components, implying the need for further localization in the convolutional architecture.

• FMA Results. To evaluate the generalizability in a scaled environment, we trained both our proposed model and the baseline in a larger dataset, i.e., the FMA dataset. Since the FMA Dataset has 8000 samples (8 times the GTZAN), both our model and the baseline were unable to capture the variance in the data. However, our model slightly outperformed the baseline with 3% accuracy, as shown in table II. We argue this phenomenon is due to the increased capacity of our model under the same constraints ($\sim$180k parameters), as our model is one block deeper than the original BBNN model. As shown the learning curves in Fig. 3, our model has a slightly lower loss in most cases. Our model confused pop with rock which is likely due to the similarity of their low-frequency components, indicating that the model might need further localization to better distinguish the difference. The model also confused the electronic with hip-hop which is also due to the similarity between both genres in almost all features as hip-hop is sometimes categorized as a sub-genre of electronic genre.

We conducted a study to understand further the ability of broadcast networks to outperform state-of-the-art music genre classifiers like those presented in [33, 34, 35, 21, 36, 37, 38, 39]. In this section, we describe our variations and discuss our findings.