[PATCH] xfree86/modes: Take rotation into account when checking mode size

[ville.syrjala at nokia.com]

From: Ville Syrjälä <ville.syrjala at nokia.com>

Assume that a mode can be used in either landscape or portrait
orientation. I suppose the correct thing to do would be to collect all
the supported rotations from the CRTCs that can be used with a specific
output, but that information doesn't seem to be readily available when
these checks are done. So just assume that either orientation is fine.

Signed-off-by: Ville Syrjälä <ville.syrjala at nokia.com>
---
 hw/xfree86/modes/xf86Modes.c |    9 ++++++---
 1 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/hw/xfree86/modes/xf86Modes.c b/hw/xfree86/modes/xf86Modes.c
index 05f4319..26db2c2 100644
--- a/hw/xfree86/modes/xf86Modes.c
+++ b/hw/xfree86/modes/xf86Modes.c
@@ -365,13 +365,16 @@ xf86ValidateModesSize(ScrnInfoPtr pScrn, DisplayModePtr modeList,
     DisplayModePtr mode;
 
     for (mode = modeList; mode != NULL; mode = mode->next) {
-	if (maxPitch > 0 && mode->HDisplay > maxPitch)
+	if (maxPitch > 0 && xf86ModeWidth(mode, RR_Rotate_0) > maxPitch &&
+	    xf86ModeWidth(mode, RR_Rotate_90) > maxPitch)
 	    mode->status = MODE_BAD_WIDTH;
 
-	if (maxX > 0 && mode->HDisplay > maxX)
+	if (maxX > 0 && xf86ModeWidth(mode, RR_Rotate_0) > maxX &&
+	    xf86ModeWidth(mode, RR_Rotate_90) > maxX)
 	    mode->status = MODE_VIRTUAL_X;
 
-	if (maxY > 0 && mode->VDisplay > maxY)
+	if (maxY > 0 && xf86ModeHeight(mode, RR_Rotate_0) > maxY &&
+	    xf86ModeHeight(mode, RR_Rotate_90) > maxY)
 	    mode->status = MODE_VIRTUAL_Y;
 
 	if (mode->next == modeList)
-- 
1.7.2.2