We propose an algorithm to combine both depth and position measurements when estimating a continuous surface. Position measurements originate from a fixed point on the surface, whereas depth measurements are determined by the intersection of the surface with a line originating from the depth sensor. Through fusion of both types of measurements, it is possible to benefit from the advantages of different sensors. The surface is obtained through interpolation of control points with splines, which allows a compact representation of the surface. In order to simplify the problem of intersecting the surface with lines originating from the depth sensor, we propose the use of polar or spherical coordinates in surface parameterization. The presented algorithm can be applied in both 2D and 3D settings and is independent of the particular choice of sensors. Our method can recursively include new information as it is obtained by using nonlinear filtering and it considers uncertainties associated with the measurements.