My volumes disappeared!

[kim0]

I have bad experience with Xmx nothing scientific, but I'd recommend you make sure the physical box has at least 2G RAM (better 4G) as mentioned in

https://help.ubuntu.com/community/UEC/CDInstall

[raymdt]

Hi,
please read this.
http://open.eucalyptus.com/wiki/Euca...KnownBugs_v2.0

try to reinstall Eucalyptus if that article don't help.
Please give a feeback.

[Ohm's Lawyer]

Well, I certainly have more than the suggested RAM. Currently running 16G on each machine. As far as the article, I seem to be doing ok. I'm not showing any errors related to space. I currently have a few TB for the volumes and only using a few gigs.

I did try modifying the xmx and xms values, and so far everything seems to be working. We'll see what happens down the road, but for now it's doing well.

Thanks guys!

[Ohm's Lawyer]

So it's happened again. It appears this is definately a Java Heap space error. When I check the cloud-debug.log on my storage controller, I get the following error:

17:08:40 DEBUG [Context:New I/O server worker #1-13] :1299024520981:Context:CONTEXT_USER:17d4872c-ece7-41e8-899f-2a7de08b85c9:admin:MessageStackHandler.handleUpstream.127
06:43:40 ERROR [SystemClock:SystemClockTimer] java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
06:44:59 ERROR [SystemClock:SystemClockTimer] java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space

But if I bump the heap up any more, it causes a world of other problems. Currently on my eucalyptus.conf I have the following set for my heap

CLOUD_OPTS="-Xms512m -Xmx512m -XX:MaxPermSize=256mm"

As stated previously,

I've currently got 16gig of RAM and 24gig of swap

So I don't think I'm running out of RAM

I'm not entirely sure what my next step should be. Any ideas? Does anyone know if there is an open ticket on this issue?

Thanks again in advance.

[estepix]

Hi I was having a similar problem.

On this Eucalyptus thread, there is a lot of info about a similar problem.

http://open.eucalyptus.com/forum/20-...e-after-reboot

At the very bottom, a couple of solutions are provided.

Here is my own recipe:
Although this is something that needs to be done every time your SC server is restarted, it seems to me faster than reinstalling the application and a clean work around to the problem. I have being trying it for a while and seems to be working perfectly. By the way, I've added the "/run/lock" part as it also seems to happen every time I restart the server.
Delete all iSCSI targets
tgtadm --mode target --op delete --tid=
Activate lvm2 volumes
vgscan
vgchange -ay
Also, after rebooting the OS we will need to run this command:
mkdir /run/lock
Otherwise we will get this error on cloud-error.log
00:25:26 ERROR [SystemUtilool-8-thread-1] com.eucalyptus.util.ExecutionException: ///usr/lib/eucalyptus/euca_rootwrap pvcreate /dev/loop6 error: /var/lock/lvm: mkdir failed: No such file or directory
File-based locking initialisation failed.
Restart SC service
/etc/init.d/eucalyptus-cloud restart

[Ohm's Lawyer]

Thanks for all of the help. Basically, since it was a recurring problem, I just scrapped everything and put up a fresh 11.04 cloud. It seems to have done the trick so far.