Abstract

Creating a musical video that contains a sequence of several clips can be a hard task. Selecting, ordering, and simultaneously considering the tension, feeling, or emotions happening in a musical piece can be difficult. This work relates the tonal tension of a musical piece with the visual and color characteristics of a set of clips. Moreover, the motion of components, the saturation level, and the colorfulness are considered in each clip. The idea is to tackle this problem as a combinatorial opti-mization problem, where the distance between tonal tension and visual features can be optimized considering different criteria. Here, a local search method is proposed and evaluated to optimize visual features considering real clips recorded with a cellphone and well-known musical pieces. Also, the K-predominant color distance between neighbor clips is minimized to avoid abrupt changes in the selected clip sequence. Results show that the proposal can successfully select and order a subset of clips relating the tonal tension with the mentioned features. Also, scenarios with different importance of the visual clip features are presented and analyzed to evaluate the flexibility of the proposed approach.