The picturesque town ties its origins to the agglomeration that developed around the hermitage of the Irish monk Ursicinus.
The "St. Peter's Gate", at the eastern entrance, has a medieval lower structure and is surmounted by a pavilion roof with a small bell tower (1552 and 1665). To the west corresponds the "Porte Saint-Paul" (Porrentruy Gate), which was completely rebuilt in 1664. The south entrance is represented by the "Porte Saint-Jean", which is at the same time the bridgehead of the bridge over the Doubs. The collegiate church is a Romanesque basilica with pillars without a transept. Under the 12th century choir is the crypt. The nave is still faithful to the Romanesque churches of Burgundy and Franche-Comté. The facade of the bell tower is a reconstruction from 1442. The southern portal is inspired by the so-called "St. Gallus Gate" of Basel Cathedral; in the tympanum, Christ surrounded by Saints Peter and Paul, seven angels and a donor; in the side niches, the Virgin with Child and St. Ursicinus. The town hall was rebuilt in 1825. The ground floor, which opens onto arcades with pillars and vaults, dates back to the previous Gothic building.