Freeing up memory for JDeveloper 12c on Windows platform

Environment / Assumptions in this post:

• Windows 7
• JDeveloper 12c
• SQLDeveloper 4.1

When I got a new desktop at work, it was one of those tiny but powerful units that sits mounted behind your monitor. It had a 128Gb SSD, which is small but should be enough for a Windows 7 Enterprise install with just a few productivity tools.

Or so I thought. It turns out that at least two if not three versions of JDeveloper, SQLDeveloper, Cygwin, plus all the usual browsers and Office suite tools takes up lots of space. Plus the projects. I was having to keep things pretty lean to stay afloat from a space point of view. I was constantly having to empty the recycle bin and tmp folders, but it was quickly becoming apparent that those actions were not going to be enough.

So after a bit of digging around, I came up with a short list of things to do that I passed on to my co-worker developers. These may not all apply to other Windows versions and some of these you will want to approach with caution.

I would install WinDirStat – very good Open Source program inspired by the KDirStat Linux utility (update: KDirStat is dead, it is now QDirStat). Make sure to run it on c: only.  It will tell you which folders are taking up how much space, which is really useful to find those hidden hogging directories.

Stuff to Delete

• If you run the Integrated Weblogic in JDeveloper, you may end up with lots of artifacts from deploying. You can try checking in your ...\AppData\Roaming folder, similar to this:
  C:\Users\jjames\AppData\Roaming\JDeveloper\system12.2.1.0.42.151011.0031\DefaultDomainYou can delete the logs and tmp folders at the very least. Don't forget that you can do this for each of your Middleware Homes.
   
• Delete from Trash Bin, and make sure it isn’t sized too big.
   
• Delete the various Temp folders, especially under c:\Windows.
   
• Uninstall programs that are not needed(!).
   
• Turn off “System Protection”. Do this at your own risk! Go to System Properties / System Protection / Configure.
   
• Look for HPROF files these are java heap dumps. They can be very big, and you can get them when for example an Oracle product does a thread dump. You can examine them with JVisualVM, or there is a great tool called MAT (an Eclipse project). They are usually under….can’t remember if it was ORACLE_HOME or if it was under ROAMING. I had a few of them.
   
• There is a big folder c:\Windows\Logs\CBS.  Turns out this takes up an artificially high amount of space, because of the way hard links are accounted for in Windows. This can be trimmed by running the “Disk Cleanup” wizard, start it up and click “Clean System Files”. It will add a few more checkboxes to the list. This can take a while.
   
• Finally, you can disable Hibernate. This probably isn’t needed for desktops. Proceed at your own risk here too. Run a cmd shell with Admin, then type:
  
  powercfg.exe /hibernate off

Web Center Content - Database Views to answer simple questions

Environment / assumptions for this post:

• Web Center Content 11.1.1.8
• Oracle DB 11.2.0.4

This started as a way to answer the question: “do we have any documents bigger than x number of Mb”.  Pretty simple question, not necessarily the easiest to answer. There are some reports and statistics available through the UCM Admin Console itself, but I wanted a simple way to provide this sort of data to another tool, say, BI Publisher.  With that, users can create their own reports and schedule them.

We actually used this sort of approach before on one of our major application projects, which was based on the old Oracle Portal 11g. By building over a dozen views, we were able to make internal tables such as:
WWV_THINGS
WWV_THINGATTRIBUTES
etc
...much more accessible. Then, by building views (and ADF View Objects) on top of those basic views, it became easy to extract information for particular screens and use cases.

UCM_FOLDERS View

First, start with a basic view which assembles folder/location information, as well as things like creator, last modified, etc. We'll call it UCM_FOLDERS.

CREATE OR REPLACE FORCE VIEW "DEV_OCS"."UCM_FOLDERS_VW" ("LEVEL","PATH","FOLDER","CREATED","CREATOR","LAST_MODIFIED","LAST_MODIFIED_BY","SECURITY_GROUP","COLLECTION_ID") AS
select level "LEVEL",
       SYS_CONNECT_BY_PATH(dcollectionname, '/') "PATH",
       dcollectionname "FOLDER",
       to_char(dcreatedate,'yyyy-mm-dd hh24:mi:ss') "CREATED",
       dcollectioncreator "CREATOR",
       to_char(dlastmodifieddate,'yyyy-mm-dd hh24:mi:ss') "LAST_MODIFIED",
       dcollectionmodifier "LAST_MODIFIED_BY",
       dsecuritygroup "SECURITY_GROUP",
       dcollectionid "COLLECTION_ID"
from collections
start with dparentcollectionid=-1
CONNECT BY nocycle prior dcollectionid=dparentcollectionid and dparentcollectionid!=-1;

Some examples of queries you could do:

select * from ucm_folders_vw where folder='jjames';
select * from ucm_folders_vw where path like '/Contribution Folders%';
select * from ucm_folders_vw where creator='jjames';


UCM_DOC_FOLDERS View

Now, we can build on that. In this case, we are interested in things like document name and size. Let's call it UCM_DOC_FOLDERS. We'll also include a few important columns like DOC_ID and COLLECTION_ID, which may be useful if we need to link to other tables.

I've limited to Primary versions only.

CREATE OR REPLACE FORCE VIEW "DEV_OCS"."UCM_DOC_FOLDERS_VW" ("DOC_ID","DOC_NAME","DOC_SIZE","FOLDER_NAME","LEVEL","PATH","CREATED","CREATOR","LAST_MODIFIED","LAST_MODIFIED_BY","SECURITY_GROUP","COLLECTION_ID") AS
select doc.did "DOC_ID",
       doc.doriginalname "DOC_NAME",
       doc.dfilesize "DOC_SIZE",
       collec.dcollectionname "FOLDER_NAME",
       folders.path_level "LEVEL",
       folders.path "PATH",
       to_char(dcreatedate,'yyyy-mm-dd hh24:mi:ss') "CREATED",
       dcollectioncreator "CREATOR",
       to_char(dlastmodifieddate,'yyyy-mm-dd hh24:mi:ss') "LAST_MODIFIED",
       dcollectionmodifier "LAST_MODIFIED_BY",
       dsecuritygroup "SECURITY_GROUP",
       collec.dcollectionid "COLLECTION_ID"
from documents doc
join docmeta meta on (doc.did=meta.did)
join collections collec on (meta.xcollectionid=collec.dcollectionid)
join ucm_folders_vw folders on (collec.dcollectionid=folders.collection_id)
where doc.disprimary=1;

Some examples of queries you could do with this:

select doc_name,doc_size,path,last_modified
from ucm_doc_folders_vw 
where creator='jjames' and doc_name like '%test%';

Using the JRebel class loading tool with ADF projects

The JRebel tool, available from Zeroturnaround, is a class loading tool. It allows java class file changes to be loaded without any redeploys or restarts; it does this by extending the class loader and instrumenting classes. It not only will load changes to method bodies, but new methods, changes to interfaces, adding fields and enum values, and even property bundle changes.

It is a "java agent" plugin, available for Java 5+. They have many plugins for various frameworks (e.g. Groovy, Hibernate, myfaces, RESTEasy, Spring, Struts, Velocity), but it is the ADF one that caught my interest. I'll be focusing on how to install on a Windows desktop with JDeveloper, using the Integrated Weblogic. Other setups can likely be accommodated with little changes.

The really cool part for ADF developers though is that is also has the ability to handle changes to things like pagedef files, Application Module definitions and View Object definitions. This obviously comes in extremely handy. It is not free, but the licenses are pretty cheap. One important thing to note: JRebel doesn't magically know that you changed a java source file - it works by comparing dates on monitored folders. This means that you have to do a 'make' on your project so the .class file is updated. Usually this is relatively fast.

Is it a new idea? No, not at all. Existing methods have been around for quite some time now, but the JRebel solution is more comprehensive.

Existing method 1: Hot Deploy
When an application server (e.g. JBoss, Weblogic’s "Fast Swap") will redeploy upon code changes. Decent sized app could be 30 to 45 seconds.

Existing method 2: Hot Swap
Since Java 1.4, you can redefine classes inside a debugger session. However, this only works for method bodies.

Existing method 3: Throwaway class loaders
E.g. Apache jackrabbit, various schemes available with many containers including Spring, ATG, WebSphere, WL. Only works for Managed Components, and has other problems.

Firstly, JRebel can work from within JDeveloper and provides simple buttons to turn on and off. However, this only works with JDev 11.1.2 and above.  We are still using earlier versions so I opted for the "standalone" version. This simply means that you get it running with some JVM parameters rather than a more point-and-click configuration.

Prerequisites:

• JDeveloper installed, 
• JRebel standalone version downloaded
• JRebel license file

Installation

1. Install the JRebel standalone product somewhere, e.g. C:\Programs\jrebel-6.1.0-nosetup.

2. Open a command window, and run the setup.

cd c:\Programs\jrebel-6.1.0-nosetup\jrebel
bin\setup.cmd

This should create a .jrebel folder in your home directory, e.g. C:\Users\jjames\.jrebel.

Activation

Put the license file in the .jrebel folder. In a cmd window, run the activate script:

cd C:\Programs\jrebel-6.1.0-nosetup\jrebel
bin\activate.cmd jrebel.lic

Modify Weblogic

Add to JAVA_OPTIONS:

-noverify -javaagent:C:\path_to_jar\jrebel.jar -Drebel.adf_core_plugin=true -Drebel.adf_faces_plugin=true

Note: you will need to boost your PermGen from default 64m to at least 128m. You may also need to boost your max heap size. If you still have problems, switch to a 64bit JVM.

-Xms512m -Xmx1536m -XX:PermSize=512m -XX:MaxPermSize=512m

Project Configuration

Apart from the JVM params, the only other configuration you really need is one rebel.xml file per JDev project that you want monitored. For your typical ADF application, that usually means a "Model" project and a "View Controller" project. You are simply pointing to the folders where the class files pagedef's, and JSP's live.

Here is a sample from one of my ADF projects. There is documentation on this here.

The Model project rebel.xml:

<?xml version = '1.0' encoding = 'UTF-8'?>
<application
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.zeroturnaround.com/alderaan/rebel-2_0.xsd"
    xmlns="http://www.zeroturnaround.com">
  <classpath>
    <dir name="C:\Projects\HelloWorldApp\ADF\HelloWorld\Model\classes">
      <include name="*.class"/>
      <exclude name=".wlsjsps/**"/>
    </dir>
    <dir name="C:\Projects\HelloWorldApp\ADF\HelloWorld\Model\src">
      <exclude name="*.java"/>
    </dir>
  </classpath>
</application>

The View Controller project rebel.xml:

<?xml version = '1.0' encoding = 'UTF-8'?>
<application
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://www.zeroturnaround.com/alderaan/rebel-2_0.xsd"
      xmlns="http://www.zeroturnaround.com">
   <classpath>
      <dir name="C:\Projects\HelloWorldApp\ADF\HelloWorld\ViewController\adfmsrc"/>
      <dir name="C:\Projects\HelloWorldApp\ADF\HelloWorld\ViewController\classes">
         <include name="**/*.class"/>
         <exclude name=".wlsjsps/**"/>
      </dir>
      <dir name="C:\Projects\HelloWorldApp\ADF\HelloWorld\ViewController\src">
         <exclude name="**/*.java"/>
      </dir>
      <dir name="C:\Projects\HelloWorldApp\ADF\HelloWorld\Model\classes">
         <include name="**/*.class"/>
         <exclude name=".wlsjsps/**"/>
      </dir>
   </classpath>
   <web>
      <link target="/">
         <dir name="C:\Projects\HelloWorldApp\ADF\HelloWorld\ViewController\public_html"/>
      </link>
   </web>
</application>

Starting Up

During the WLS startup, you should see some startup messages like the following:

2015-03-03 14:52:12 JRebel: ############################################################# 2015-03-03 14:52:12 JRebel: JRebel Legacy Agent 6.1.0 (201502250834) 2015-03-03 14:52:12 JRebel: (c) Copyright ZeroTurnaround AS, Estonia, Tartu. 2015-03-03 14:52:12 JRebel: 2015-03-03 14:52:12 JRebel: Over the last 1 days JRebel prevented 2015-03-03 14:52:12 JRebel: at least 0 redeploys/restarts saving you about 0 hours. 2015-03-03 14:52:12 JRebel: 2015-03-03 14:52:12 JRebel: Licensed to Jonathan James 2015-03-03 14:52:12 JRebel: ############################################################# .... <11-Mar-2015 3:45:40 o'clock PM EDT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STANDBY> <11-Mar-2015 3:45:40 o'clock PM EDT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING> 2015-03-11 15:45:50 JRebel: Directory 'C:\CVS_RCeds\RCeds\ADF\RCeds\ViewController\adfmsrc' will be monitored for changes. 2015-03-11 15:45:50 JRebel: Directory 'C:\CVS_RCeds\RCeds\ADF\RCeds\ViewController\classes' will be monitored for changes. 2015-03-11 15:45:50 JRebel: Directory 'C:\CVS_RCeds\RCeds\ADF\RCeds\ViewController\src' will be monitored for changes. 2015-03-11 15:45:50 JRebel: Directory 'C:\CVS_RCeds\RCeds\ADF\RCeds\Model\classes' will be monitored for changes. 2015-03-11 15:45:50 JRebel: Directory 'C:\CVS_RCeds\RCeds\ADF\RCeds\ViewController\public_html' will be monitored for changes. 2015-03-11 15:45:50 JRebel: Directory 'C:\CVS_RCeds\RCeds\ADF\RCeds\Model\classes' will be monitored for changes. 2015-03-11 15:45:50 JRebel: Directory 'C:\CVS_RCeds\RCeds\ADF\RCeds\Model\src' will be monitored for changes.

When a class or other file is reloaded, you'll see something like:

2015-03-11 16:04:32 JRebel: Reloading class 'edu.rcpsc.eds.view.backing.ItemWorkList'.
or:
2015-03-11 16:13:31 JRebel-Mojarra: Reloading configuration

Development Tools - beyond JDeveloper and SQLDeveloper

Environment / assumptions for this post:

• Windows 7+ desktop

Here are a few tools I've been using day to day that have come in very useful during normal development activities. I'm sure there are many alternatives to each of these, but these are either my favourites or maybe just what I've gotten used to.

Note: for useful programs equivalent to those below, checkout my post on the Linux Desktop in the workplace.

Cygwin

Cygwin is a collection of GNU and Open Source tools - basically Linux on Windows. If you have to use Windows for development and your servers are Linux based, then Cygwin can be greatly helpful. I love to use it for troubleshooting remote JVMs with JConsole, and use it for ssh shell access to the servers.

Cygwin terminal on Windows 7 - remotely connected to Linux
server with SSH and showing JConsole

Notepad++

Runner-up: Textpad
This is a great text editor - free, governed by GPL. Every day I take advantage of features like:

• all the usual multiple tabs, syntax highlighting, multiple encodings, etc.
• good regex support
• if you close the program, you can restart with all your files/tabs intact even if they weren't saved 
• even for searching the contents of files in a directory hierarchy

Notepadd++ - multiple tabs, and lots more

WinMerge

This is an Open Source diff / merge tools for Windows. Very nice, you can compare individual files or whole directories, ignoring certain files if you like. You can also just cut&paste into left/right windows.

WinMerge - diff / merge tool for Windows

WinDirStat

Open Source directory statistics viewer. Similar to KDirStat or QDirStat on Linux. Comes in really handy when you have a small SSD drive and are running out of space.

WinDirStat - directory statistics viewer

DJ Java Decompiler

Sometime you have to go to greater lengths than normal to troubleshoot issues. In those cases, being able to decompile classes is very useful, and this is probably the best program to do it. An earlier release is free, but the latest is $20US although it has a trial version.

DJ Java Decompiler

LdapAdmin

The Ldap Admin tool is also free, and allow you to connect to any Ldap server including Oracle OID. It is a nice way to search and verify things if OIDDAS isn't available, or you don't feel like using command-line tools.

Ldap Admin tool - connect to Ldap directories

VLC Media Player

Every developer needs to drown out their co-workers occasionally.  :)  For such times, VLC comes in handy - it is open source and multi-platform.

VLC Media Player

Snipping Tool

Believe it or not, even some people who have been using Windows for years may not realize that a very useful accessory tool is included with Windows - the Snipping Tool. There are better tools out there but this one is right there ready to go.

Microsoft's Snipping Tool

IrfanView

This is freeware, so free for personal or educational use but otherwise $12US. I suppose there are newer free image viewers, but I've been using this for a long time and I'm used to it. Very small and fast, easy to do simple manipulations. It is found here.

IfranView - nice little fast image viewer

ADF 12.2.1 - ExportCollectionActionListener exports column headings incorrectly after ordering columns

Update March 2017: this bug is still NOT FIXED. The Oracle Bug does not have a Fix Version.

Environment / assumptions for this post:

• JDeveloper 12.2.1.0
• Windows 7
• Custom Fusion Web Application

This is admittedly a fairly low priority issue for us, but under the right circumstances it could be a big deal - if you export data from tables a lot.

In a nutshell: if you reorder a column in an af:table and then export the data via ExportCollectionActionListener, the data will be correct but the column headers will be the original header order.

To illustrate, I created a simple Test Case to submit to Oracle. 

Step 1 - simple table using a ViewObject that points to a simple basic database Table

Original table of data with Export button

Step 2 - Export the data and open in Microsoft Excel, looks fine

Data looks fine in Microsoft Excel

Step 3 - Re-order a column by dragging

Drag the order of columns - swap Firstname and Lastname

Step 4 - Export again - column headers wrong!

Second Export to Excel - data is correct but column headers are same as original

For reference, the Oracle BUG is #22791014.

Troubleshooting Fusion Middleware applications - configuring SSH on Windows

Environment / assumptions for this post:

• Windows 7 desktop
• Oracle Enterprise Linux 7.2
• Cygwin with SSH packages

My work machine was and still is on the venerable Windows 7, but this applies to Windows 8/10 just as well. All of our Dev / Test / Production environments are Oracle Enterprise Linux based, and we have many applications deployed to Weblogic 12c.  Although we can sometimes rely on Nodemanager and Enterprise Manager for restarts and things like checking log files, it is sometimes easier to do things manually from the command line. Occasionally some creative use of 'grep' is invaluable in finding specific issues.

In any case, there are many ways to configure ssh for this, but I'll just focus on two: the easy way and the long way.

Easy Way - PuTTY

A simple install and go client like PuTTY is very quick and easy. Download here. Also available as part of the download are scp and sftp clients. There's also MobaXterm and many others.

Longer Way - Cygwin

Again, not so much hard but longer - although worth it in the end! Cygwin is a bunch of Open Source tools that give Linux functionality to Windows. You download an installer, then selectively choose the packages you like. There are tons of websites and blogs that describe in graphic detail how to install and configure Cygwin. Apart from basic ssh sessions, you can also do things like start an X server and launch programs from a remote server i.e. using your own monitor as the display. This can be very helpful - see my other post on this.

First, download and install Cygwin. Search for and select the openssh package and finish the install.  Any required dependencies will automatically be chosen for you.  Lots of default base install stuff will come along as well - things like bash, less, grep, etc etc.
Note: you can go back to the cygwin install directory at any time and re-run the installer to install additional packages.

Cygwin initial setup - search for ssh and select the openssh package.

Under your cygwin folder, you'll have a home directory. Mine looks like:

c:\cygwin64\home\jjames

To start a session, just find the Cygwin Terminal under the Windows menu if you chose to create it there. The first time you run it, the usual login scripts will be created such as .profile and .bash_rc. You can then ssh to one of your servers. The very first time you do so, you will get a message about authenticity - just type yes and the file "known_hosts" under a new folder .ssh will be created.

$ ssh oracle@hostname
The authenticity of host 'hostname (192.168.1.1)' can't be established.
RSA key fingerprint is SHA256:ABCDEFT9OlFlKy9YIv4Fsj0RXCldRbv4MKMxab8P5iw.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hostname,192.168.1.1' (RSA) to the list of known hosts.
oracle@hostname's password:

At this point, you can stop - no more is required. 

There are other cool things you can do though, such as generating a public/private key pair, and other misc config that will help with things like X forwarding.  See my post about running a remote JConsole on your local X server.

Public/private key pair

Next, you will want to generate a public/private key pair. This will allow you to embed the key in the authorized_hosts file in your home directory on a remote server without having to enter a password, or by entering the same password i.e. you could use the same password for many servers without changing your actual password on each machine. So you do this by running ssh-keygen as follows:

$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/jjames/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/jjames/.ssh/id_rsa.
Your public key has been saved in /home/jjames/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:us/XYZABCDEFPW12345ZAeW+aUBKpBunYxLix+d58KI jjames@myhost
The key's randomart image is:
+---[RSA 2048]----+
|                 |
|  x  s      .    |
|       o   o     |
|  . . o o o o .  |
| . e . *So = . . |
|  . + B.t . = .+ |
|   . j.=  . .=+o+|
|      -oo.o.Boo+o|
|     ..+o..o.oo..|
+----[SHA256]-----+

jjames@myhost ~
$

Two files are generated: id_rsa and id_rsa.pub which are the private and public keys respectively. Add the value in the public key file to each remote server in ~/.ssh/authorized_hosts.

Misc Config

Finally, you can optionally put other configuration in the config file. My config file contains many entries that look like this:

host dev
 hostname development.company.com
 user oracle
 ForwardX11Trusted yes
 ForwardX11 yes
 ServerAliveInterval 300 
 ForwardAgent yes

An explanation is as follows:

host dev - in my case, 'dev' is just an alias, it can be whatever you want.
hostname development.company.com - the server
user oracle - the server login
ForwardX11Trusted yes - set this for X11 forwarding
ForwardX11 yes - set this for X11 forwarding
ServerAliveInterval 300 - seconds before sending a null packet to keep the connection alive
ForwardAgent yes - this is about forwarding authentication when you log in from one machine to another. There is a good guide here.