This paper presents a specific architecture based on a multilevel formalism to control either autonomous or teleoperated robots that have been developed in the laboratory vision and robotics. This formalism separates precisely each robot functionalities (hardware and software) and provides a global scheme to position them and to model data exchanges among them. Our architecture was originally built from the classical control loop. Two parts can thus be defined: the Perception part which manages the processing and the models construction of incoming data (the sensor easurements), and the Action part which manages the processing of controlled outputs. These two parts are divided in several levels and depending on the robot, the control loops that have to be performed are located at different levels: from the basic one (i.e. level 0) composed by the jointed mechanical structure and level 1 which performs actuators servoing, to the highest one (i.e. level 5) which manages the various missions of the robot. The higher the level is, the shorter the loop reaction time has to be. Each level clusters, for their respective part, specific modules which perform their own goals. This general scheme permits to integrate different modules issued from various robot control theories. This architecture has been designed to model and control autonomous robots. Furthermore, a third part, called "teleoperated part", can be added and structured in levels similarly to the two other parts. Distant from the robot, this teleoperated part is managed by a human operator who controls the robot at various levels: i.e. from the first level (the basic remote control) to the upper one (where the operator only controls the robot mission). Hence, this architecture merges two antagonist concepts of robotics, i.e. teleoperation and autonomy, and allows a sharp distribution between these two fields. Finally, this architecture can be used as a main frame to build its own control architecture, using only a few clusters with dedicated modules. Some examples and experimental results are given in this paper to validate the proposed formalism.