facebook

10.1 upgrade runs out of memory building workspace

  1. MyEclipse IDE
  2.  > 
  3. Installation, Configuration & Updates
Viewing 15 posts - 1 through 15 (of 22 total)
  • Author
    Posts
  • #326187 Reply

    mongonv
    Participant

    Hey All,
    I just installed the full 10.1 all-in-one to upgrade projects we have built using 8.6 and when 10.1 tries to build the workspace it gets between 70-80% and runs out of memory.

    I update the ini to use the same memory configurations I have in the 8.6 which are as follows:

    -vmargs
    -Xms256M
    -Xmx1024M
    -XX:ReservedCodeCacheSize=128m
    -XX:PermSize=128m
    -XX:MaxPermSize=256m

    I have the cache monitor turned on and it shows the 10.1 quickly bumping up against the upper range of over 1000M when building and then it comes up with the memory 5% low window with the default suggestions, which we had to improve on for 8.6 but are of no help in 10.1.

    The projects are pretty large, well over 1 million lines of code each and build fine in 8.6, but of the two different projects I have tried so far, both fail to build under 10.1.

    I did remove a number of the server support packages we do not use, with the config center. When I did get one project to where I could edit it, it was very slow in doing simple editing like going to the home of a line, double clicking to select text, and pasting text into the selection.

    I am running on a 6 core processor with 8 gig or ram, so the machine is fast and performs VERY well with 8.6.

    What else can be done to get this new version to improve its performance and be able to build the workspace?

    Thanks for any advice,
    Ed

    #326227 Reply

    support-joy
    Member

    Ed,

    Sorry to hear you are running into memory related issues. You have not included installation details. I am not sure what OS you are using and also if you are on 32 or 64 bit? I would recommend you take a look at this FAQ on trouble shooting memory and performance related issues – https://www.genuitec.com/forums/topic/troubleshooting-memory-amp-performance-issues/

    I am guessing that your current issue could be related to JavaScript. From ME 9.1 and later, we have introduced several JavaScript code analysis mode. I would recommend you turn off these features if not required by your current projects and give it a shot.

    My apologies for the inconvenience caused. Let me know how this works for you.

    #326234 Reply

    mongonv
    Participant

    Sorry about forgetting the system info, I am on Windows 7 64-bit and that is the same OS as the 8.6 was running under.

    I checked the Javascript settings and I was already using passive mode, so I turned it off completely. I had done that with 8.6 as the editor was not really as functional as I needed, so unless it has improved significantly since 8.6, will leave it turned off.

    I did some playing with the memory and by increasing the max memory from 1gig to 2gig and it will now complete the builds, but when I have my other typical working apps open it leaves me very little available memory on the system. When 10.1 does the build it does run the memory usage right up against the 2 gig mark in the heap monitor, which is just about double the amount it used under 8.6.

    Any other thoughts would be appreciated.

    #326255 Reply

    support-joy
    Member

    Ed,

    Thank you for sharing your system information. Instead of bumping up the memory settings to double, I would recommend you increase it bit-by-bit in 256 MB increments. You might find a lower memory setting which works for you.

    #326463 Reply

    mongonv
    Participant

    Even with the 2gig allocated to the myeclipse.ini I am still getting out of memory problems in the 10.1 when I am doing things like generating web services or doing clean builds. 8.6 works fine with 1024Meg of memory allocated and responds faster on the same machine.

    I had to return to using 8.6 since 10.1 just does not seem to be able to do the job. In trying to use it for our main project, I was spending more time waiting for it to error out from lack of memory or just do simple edit tasks then I was creating new code.

    Just basic editing is tediously slow in doing basic tasks like selecting text and pasting text. Is there some magic setting that is needed to get reasonable performance out of 10.1?

    #326507 Reply

    Brian Fernandes
    Moderator

    Ed,

    As far as performance is concerned, the only place where 10.x may actually be worse than 8.x is in terms of the JavaScript performance. The reasons for this are explained in the memory troubleshooting post Joy pointed to earlier. In other areas, MyEclipse 10 is significantly better than 8.x as far as performance is concerned.

    When the low memory warning does come up, could you let us know exactly what warning is displayed? It should mention the type of memory pool that is running out (Survivor, Eden Space, Old gen, etc). Have you ever ignored this warning (and this is when the you see bad performance as far as copy/paste, typing is concerned) or do you restart immediately when you see this error? Are there any other messages / error logs that result from the memory problem?

    What is the composition of your project? You mentioned a million lines of code – a very rough estimate of how that is split across Java, JSP, HTML or JS would help.

    When I did get one project to where I could edit it, it was very slow in doing simple editing like going to the home of a line, double clicking to select text, and pasting text into the selection.

    Which editor(s) do you experience this problem with?

    If you can answer these questions, we can take things further, perhaps to a live support session if possible – if there are problems, I’d like to fix them for the upcoming 10.5 release.

    Sorry for the inconvenience caused and thank you for reporting your experience.

    #326509 Reply

    mongonv
    Participant

    Hey Brian,
    The main project is 100% Java, no JS, JSP, or HTML in the main project, there are two pretty much static secondary projects in the workspace: a BIRT report web service project and one that does contain JS but is simply updated and used to build a WAR, no editing of it occurs within MyEclipse.

    The editor being used is the default Java editor and I think I have turned off the JS stuff suggested by Joy earlier, so I don’t think JS is the issue in my installation, but could be missing something.

    When the memory problem occurs I tried to get what was running to finish and if it is just hung I kill ME and restart it. As these issues have been so consistent I have had to go back to using 8.6 for my daily work.

    I just duplicated the issue by starting up the 10.1 workspace, it did a building workspace that took significantly longer than the 8.6 does, did a regen of a web service, and when that finished started a clean on the OF-WsMain project only and that is when the mem problem came up. I have attached a screen shot showing the error and the main application with the heap monitor on the task bar showing the amount of memory usage. I have 2gig of memory allocated for the application and here are my current settings from MyEclipse.ini:

    -vmargs
    -Xms256M
    -Xmx2g
    -XX:ReservedCodeCacheSize=128m
    -XX:PermSize=128m
    -XX:MaxPermSize=256m
    -Dosgi.nls.warnings=ignore

    I responded to the message to Close and it did close after about 1.5-2 minutes.

    Let me know if there is anything else you need from my installation.

    Attachments:
    You must be logged in to view attached files.
    #326530 Reply

    Brian Fernandes
    Moderator

    Ed,

    Thanks for the detailed response. For a simple Java project to cause OOM issues is odd.

    1) As a sanity check, can you close the other two projects and restart your workspace. Then clean build the OF-WsMain project – can you reproduce the error?

    2) If not, regen the webservice – could you tell us exactly what wizard are you using? Are you going from WSDL to Java or bottom up from Java to WSDL + web service?). I wonder if there is something awry in the webservice generation and if that is causing the memory issues (I doubt it though).

    3) Have you installed any additional plugins?

    4) Can you send me the .project and the .classpath files of the Java project?

    5) What JDK is being used for the Java project (expand the project in the Package Explorer to confirm) – is it the 1.6 JDK that ships with MyEclipse? Is this same JDK being used with your project in 8.6 too?

    #326625 Reply

    mongonv
    Participant

    Hey Brian,
    1) I removed the other projects and did the clean build and then gen’d the main web service and it ran out of memory again on the web service generation.

    2) I am using the JAX-WS – Create web service from Java class (Bottom-up scenario)

    3) I have the following plugins installed above the defaults:
    – JAutoDoc
    – Subversive – Plug-in (8 related)
    I also removed a number of unused server packages (WebLogic, Oracle server, etc. that are not related to our project) from the default installation via the Configuration Center. This was also done in 8.6.

    4) See the attached zip

    5) JRE System Library in 10.1: Sun JDK 1.6.0_13
    JRE System Library in 8.6: Sun JDK 1.6.0_13

    I also noticed that when I did a clean build it gets a problem error listed in a html file in the webroot area that the 8.6 build does not generate. I have turned off validation for html in the preferences but it is still checking it and reporting an error in the html.

    A couple stats on performance, with 2gig on 10.1 and 1 gig on 8.6 I clocked the following times on the clean build and re-gening one of the smaller web services in both versions:

    10,1:
    Clean build: 1:04 minutes
    Gen service: 1:10 minutes

    8.6:
    Clean build: 0:58 minutes 10% faster
    Gen service: 0.56 minutes 20% faster

    I did upgrade the memory in my system to 16 gig so I could allocate 4 gig to ME. With that amount allocated I am able to do a clean build and a service gen without it running out of memory. The heap monitor shows it spikes to about 2.7 gig of memory usage during each of the two operations, compared to about 0.85 gig in 8.6. Just wanted to pass that on in case it helps.

    Let me know if there is anything else you need.

    Thanks,
    Ed

    Attachments:
    You must be logged in to view attached files.
    #326637 Reply

    Brian Fernandes
    Moderator

    Ed,

    From your answers, it appears that simply clean building your workspace does not cause the problem, but actually generating the web services does? Also, you mentioned a “small” webservice later, so I assume the service you typically generate bottom up is quite large? If yes, how many lines of code (or how many methods) do you have in the class you send into the bottom up process?

    If you can say, at what point in the web service generation (exactly) does it run out of memory? Does it generate the artifacts and then run out of memory in the build stage? Or do you get no / incomplete out put from the generation process.

    As far as your other plugins are concerned, I am concerned about the JAutodoc. I do not know enough about it now to say for sure that it could cause issues, but if it monitors generated / new code to add Javadoc and if the results of the code generation process are large, this could cause problems.

    Did you have this plugin installed in 8.6 too. If so, was it the same version that is being used in your 10.x install? I would appreciate it if we could rule this out – could you remove JAutodoc and repeat the test – simply clean build and then run the web service generation?

    Thanks.

    #326666 Reply

    mongonv
    Participant

    The larger web service is about 15,100 lines and the smaller one is about 7,700 lines. The number of methods is proportionally the same.

    In testing today, it only seems to die now if I build both web services and it does not matter if I do the larger or smaller first.

    The memory alert seems to come up during the rebuilding that happens in the regen process.

    Jautodoc does not actually do anything automatically on its own, only when triggered by the user and within the scope specified by the user. I tried removing it from the project and it did not make any difference in the problem.

    Both 8.6 and 10.1 are using Jautodoc 1.9.0.

    Thanks,
    Ed

    #326695 Reply

    Brian Fernandes
    Moderator

    Ed,

    1) When you removed JAutodoc from the project – did you uninstall it or simply prevent it from being associated with the project?

    2) The only other possible issue that comes to mind is 32-bit vs 64-bit. With 8.6, we only had a 32-bit option – so even when it runs on a 64bit system, it would run as a 32-bit application.

    Your 10.1 installation is a 64-bit installation and 64-bit apps could take 30 to 40% more memory as compared to their 32-bit variants. So an increase in consumed memory is expected, but if 8.6 worked with 1G, 10.1 with 64-bit should certainly work with the 2G that you are allocating now and the 2.7 G it expands to does seem excessive.

    When you do run out of memory with the heap set at 2G, does the warning message still mention PS Old Gen? If ignored, are you still experiencing the performance degradation?

    One possible experiment, if you have the time is to re-run your 10.1 installer and install *another* instance, but this time, choose 32bit. You can then run with your 8.6 settings to see if the 32 vs 64 bit change really makes a difference.

    #326709 Reply

    mongonv
    Participant

    Hey Brian,
    I had completely removed Jautodoc from the software center .

    I did your suggestion of installing a 32 bit of 10.1 with 1 gig like the 8.6 and it seems to compile in 1 gig with no issues. Its good that it works with 1gig as it does not seem to allow going above that, it gets a message that states:

    “Failed to create the Java Virtual Machine.”

    This occurs when I try to use “-Xmx2g” in the myeclipse.ini.

    Does 32 bit not allow more than 1 gig of memory?

    So it seems that the 64 bit version is well over the 30-40% more memory you had mentioned. This is unfortunate and it seems to actually run faster in the 32 bit mode, did not run timed trials but seemed faster.

    The only other issue I am still having is that a clean build still gets a validation error in an HTML file, but I have the validators unchecked in the MyEclipse / Validation preferences. Can you tell me how to stop it from validating HTML?

    Thanks for all your help,
    Ed

    #326712 Reply

    Brian Fernandes
    Moderator

    Ed,

    For 32-bit, I have found that the max is about 1.2 to 1.3GB. This has nothing to do with MyEclipse, it is a function of both Windows and the JVM. This thread might shed more light on the subject: http://stackoverflow.com/questions/2457514/understanding-max-jvm-heap-size-32bit-vs-64bit

    It is good to know the switch to 32-bit fixed the memory consumption, but very surprising indeed to see 64-bit use more than twice the memory. We’ll try to look into this, though I doubt there is anything we can do about it.

    Out of curiosity, what processor are you using?

    As far as the HTML validator is concerned:
    a) What is the error being reported, exactly?
    b) Which validators have you unchecked? Could you please enumerate them precisely as there are a couple of them and the names are very important.
    c) If it is just a single file, you could try to right click the file and choose MyEclipse > Exclude from Validation which should prevent all validation in that file.

    #326714 Reply

    mongonv
    Participant

    Hey Brian,
    Thanks for the info.

    On the HTML validation:
    1) gets a error and multiple copies of warnings:
    Syntax error on token “<“, delete this token left.html
    Missing semicolon

    I am also getting the missing semicolon on a jsp file and have it unchecked also.

    What I have unchecked for validation is as follows:

    In Windows/Preferences/MyEclipse/Validationi both manual and build are unchecked for the following:
    HTML Syntax Validator
    JSP Content Validator
    JSP Syntax Validator
    XML Validator

    On the project properties under MyEclipse/Valitation:
    Checked Enable project specific settings (makes no diff if I use these settings or not has not gotten rid of the issues.)

    Unchecked both manual and build of the same ones as above. Under the “…” button in the Exclude group I have added the file extension of “html” and add the folder WebRoot that contains the html file. Neither of these has impacted the issue of it not doing the error and warnings in the Problems view.

    The removing the checks on manual and build worked to resolve the issue in 8.6, but do not seem to have any impact in 10.1.

    I did the right click and exclude from validation, will this stick when the file is distributed to another system?

    Any thoughts would be appreciated, cause we are getting ready to release this to clients and cannot have the errors showing up on their dev systems every time it validates.

    Thanks for all your help,
    Ed

Viewing 15 posts - 1 through 15 (of 22 total)
Reply To: 10.1 upgrade runs out of memory building workspace

You must be logged in to post in the forum log in