Model order reduction has become an inescapable tool for the solution of high dimensional parameter-dependent equations arising in uncertainty quantification, optimization or inverse problems. We focus on low-rank approximation methods, in particular on reduced basis methods and on tensor approximation methods. For projection based methods, we propose preconditioners built by interpolation of the operator inverse. We rely on randomized linear algebra for the efficient computation of these preconditioners. We also address the problem of approximating vector-valued or functional-valued quantities of interest. For this purpose we generalize the "primal-dual" approaches to the non-scalar case, and we propose a new method for the projection onto reduced spaces based on a saddle point formulation. For the low-rank approximation of tensors, we propose a minimal residual formulation with ideal residual norms. The proposed algorithm, which can be interpreted as a gradient algorithm with an implicit preconditioner, allows obtaining a quasi-optimal approximation of the solution with respect to a specified norm.