1. Copy the file /opt/netiq/idm/apps/tomcat/webapps/osp/jsp/inc_common_java.jsp to jsp folder in osp-custom-resource.jar
2. Change these properties in this copied file. All are set to 300 by default. That is why the image size is limited to that. Increase them as per your need
int LOGIN_PRODUCT_IMAGE_WIDTH_MAX = 300;
int LOGIN_PRODUCT_IMAGE_HEIGHT_MAX = 300;
int AUTH_PRODUCT_IMAGE_WIDTH_MAX = 300;
int AUTH_PRODUCT_IMAGE_HEIGHT_MAX = 300;
3. Update the jar file to include this new jsp file
4. Remove the localhost directory, copy this file to tomcat/lib and restart tomcat.