drm/i915: Flush all writes before suspend

As we have already suspended the device, this should be a no-op except
for marking that all writes are indeed complete. The downside is that
we then have to walk all the lists of objects for what should be a no-op
(in some cases they will be mmio read to ensure the GGTT writes are
indeed flushed, and clflushes to ensure that cpu writes are in memory).

It seems prudent and the safer course for us to ensure all writes are
flushed to memory before suspend.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20180601144125.18026-1-chris@chris-wilson.co.uk

drivers/gpu/drm/i915/i915_gem.c

@@ -5071,6 +5071,13 @@ err_unlock:
 
 void i915_gem_suspend_late(struct drm_i915_private *i915)
 {
+	struct drm_i915_gem_object *obj;
+	struct list_head *phases[] = {
+		&i915->mm.unbound_list,
+		&i915->mm.bound_list,
+		NULL
+	}, **phase;
+
 	/*
 	 * Neither the BIOS, ourselves or any other kernel
 	 * expects the system to be in execlists mode on startup,
@@ -5091,6 +5098,13 @@ void i915_gem_suspend_late(struct drm_i915_private *i915)
 	 * machine in an unusable condition.
 	 */
 
+	mutex_lock(&i915->drm.struct_mutex);
+	for (phase = phases; *phase; phase++) {
+		list_for_each_entry(obj, *phase, mm.link)
+			WARN_ON(i915_gem_object_set_to_gtt_domain(obj, false));
+	}
+	mutex_unlock(&i915->drm.struct_mutex);
+
 	intel_uc_sanitize(i915);
 	i915_gem_sanitize(i915);
 }