Butter, ghee and olive oil are no doubt healthy fat sources. However, if your primary concern is weight loss, sticking to one fat source would be better, depending on your preferences, lifestyle and overall food consumption.
Both ghee and butter are derived from dairy. Therefore, if you are someone who follows Veganism, sticking to oils, or nut butter would be more ideal. However, at the same time, both ghee and butter make good choices to include on a Keto diet.
As for the calorie count, butter is the heaviest calorie-dense source of all, since it is mostly composed out of fat. The fat intake is also similar in all three. What would be rather ideal is to look for the nutritional aspects of all fat sources, as they could help drive faster weight loss, and serve additional health benefits.
Of the three, incorporating olive oil would be best suited, while the other two fat sources can be added to the diet, in limited quantities.