The ubiquitous Hofstadter butterfly describes a variety of systems characterized by incommensurable periodicities, ranging from Bloch electrons in magnetic fields and the quantum Hall effect to cold atoms in optical lattices and more. Here, we introduce nonlinearity into the underlying (Harper) model and study the nonlinear spectra and the corresponding extended eigenmodes of nonlinear quasiperiodic systems. We show that the spectra of the nonlinear eigenmodes form deformed versions of the Hofstadter butterfly and demonstrate that the modes can be classified into two families: nonlinear modes that are a 'continuation' of the linear modes of the system and new nonlinear modes that have no counterparts in the linear spectrum. Finally, we propose an optical realization of the linear and nonlinear Harper models in transversely modulated waveguide arrays, where these Hofstadter butterflies can be observed. This work is relevant to a variety of other branches of physics beyond optics, such as disorder-induced localization in ultracold bosonic gases, localization transition processes in disordered lattices, and more. © IOP Publishing Ltd and Deutsche Physikalische Gesellschaft.