What version is it? If tasksel isn't working too well for you try this:
You can stop and permanently "switch off" LAMP (well the AM* component) by running:
update-rc.d doesn't check if the links exist, so make sure each command has more than one line of output (you might assume that it worked as it says "Removing any system startup links for blah")
sudo invoke-rc.d mysql stop
sudo invoke-rc.d apache2 stop
sudo update-rc.d -f apache2 remove
sudo update-rc.d -f mysql remove
it should look like this :
If you want to uninstall the AMP constituents :
Removing any system startup links for /etc/init.d/apache2 ...
Removing any system startup links for /etc/init.d/mysql ...
You may need to run sudo apt-get autoremove after that to remove the unused packages (Just remember to carefully read what autoremove is suggesting be removed)
sudo apt-get --purge remove mysql-common apache2* php5-common
Alternatively, based on https://help.ubuntu.com/community/ApacheMySQLPHP you can try :
This is a surgical removal, what i suggested was more of a carpet bomb. Either way should work fine
sudo apt-get --purge remove apache2 apache2-mpm-prefork apache2-utils apache2.2-common libapache2-mod-php5 libapr1 libaprutil1 libdbd-mysql-perl libdbi-perl libmysqlclient15off libnet-daemon-perl libplrpc-perl libpq5 mysql-client-5.0 mysql-common mysql-server mysql-server-5.0 php5-common php5-mysql