Is the following generalization of the Caratheodory theorem true?

The colorful Carathéodory theorem (Bárány, 1982) considers $d+1$ "colors" $X_1,ldots,X_{d+1}subseteq mathbb{R}^d$, and a point $x$ in the convex hull of each color ($xin text{conv}(X_i)$ for each $iin[d+1]$). It says that there exists a set of $d+1$ points of distinct colors, such that $x$ is in their convex hull.

Suppose that, instead of a single point $x$, there are $d+1$ points $x_1in text{conv}(X_1),ldots,x_{d+1}in text{conv}(X_{d+1})$. Let $bar{x} := (x_1+cdots+x_{d+1})/(d+1) = $ the average of these points. Is it true that there exists a set of $d+1$ points of distinct colors, such that $bar{x}$ is in their convex hull?

Note that the Bárány’s theorem is implied by this claim.

^{The motivation comes from this paper: https://dx.doi.org/10.1007/s00493-019-4019-y it says that a special case of Theorems 1.6 and 1.7 can be proved the the colorful Carathéodory theorem, and I wondered if the general case could be proved by a generalization of it.}