clk: Only recalculate the rate if needed

We don't really need to recalculate the effective rate of a clock when a
per-user clock is removed, if the constraints of the later aren't
limiting the requested rate.

This was causing problems with clocks that never had a rate set before,
as rate_req would be zero. Though this could be considered a bug in the
implementation of those clocks, this should be checked somewhere else.

Fixes: 1c8e600440c7 ("clk: Add rate constraints to clocks")
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Peter De Schrijver <pdeschrijver@nvidia.com>
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Signed-off-by: Michael Turquette <mturquette@linaro.org>

drivers/clk/clk.c

@@ -2664,7 +2664,10 @@ void __clk_put(struct clk *clk)
 	clk_prepare_lock();
 
 	hlist_del(&clk->child_node);
-	clk_core_set_rate_nolock(clk->core, clk->core->req_rate);
+	if (clk->min_rate > clk->core->req_rate ||
+	    clk->max_rate < clk->core->req_rate)
+		clk_core_set_rate_nolock(clk->core, clk->core->req_rate);
+
 	owner = clk->core->owner;
 	kref_put(&clk->core->ref, __clk_release);