The right and left anteromedial temporal lobes have been shown to participate in emotion processing. The aim of the study was to further address their role in music emotion perception/recognition, and assessment by two emotional determinants, i.e., arousal (relaxing versus stimulating aspects) and valence (pleasantness degree). Epileptic patients with right or left anterior mesio-temporal resection (including the amygdala), and control subjects were presented with happy musical (chosen highly stimulating) or sad excerpts (chosen to be relaxing), that were either consonant (pleasant) or dissonant (unpleasant). The patients demonstrated an abnormal perception of dissonant music disregarding of the side of the resection; thereby confirming the role of the parahippocampal gyrus in the perception of unpleasantness. Moreover, the pleasantness of musical excerpts, in particular the happy consonant ones, was overestimated by patients with right temporal damage. In contrast, the arousal rating for happy consonant excerpts was reduced only in the group with left-resections. This modified perception of arousal might be related to the decreased ability of those patients to recognize happy and sad music. Indeed, both right and left temporal resections impaired sadness recognition, whereas happiness recognition was only reduced by the left-resections. The main result was that for the first time, the mesio-temporal structures were demonstrated to be asymmetrically involved in positive musical emotion recognition.