RMAN Incrementally Updated Backups

In this post I will explore the RMAN Incremental Backups. Incremental backups are used best in Data Warehouse environments where there is a lot of data to backup and keeping multiple copies on the disk takes space.


There are few alternatives Oracle provides to optimize backups of large databases and the incrementally updated backups strategy is one of them. Actually is the Oracle recommended backup policy!

You can NOT use OS methods to take Incremental Backups. Only RMAN will let you take Incremental backups.

The incrementally updated backups allow you to apply database changes incrementally (i.e. every day) to an image copy (not zipped) full backup! That is you update the level 0 full backup with new data incrementally all the time. This way you don't have to keep old full backups, spread over days, separately with their individual archive logs and controlfiles. Even you don't have to take full backups periodically any more! You only take one full backup (level 0) at the beginning and you keep on updating it every day incrementally with the new changes in the database. By applying the changes to the last backup copy, you are bringing the backup copy forward in time. This is also called rolling forward the backup copy.

In my case I used this backup methodology to reduce the ammount of disk space used for backups, as I didn't need to keep and individual full backup of the datawarehouse for each day. I only took one full backup (level 0) at the beginning. And my retention policy was thereafter for 7 days. In this case the retention policy is the duration of time I kept the incremental backups before applying them on the full backup.

To improve the performance of incrementally updated backups another feature called Change Tracking has also been introduced in Oracle 10g. This lets you record the changes in the blocks of datafiles in a separate datafile in the database called Change Tracking File. Then when the time for backup comes, RMAN reads the Change Tracking File to find out the changes which happened to the database instead of scanning whole datafile. This makes the backup much more faster.

So, before running the incrementally updated backup backup script, I enable change tracking in the database like this:

1. First check to see if change tracking is already enabled by querying the DBA view like this:

SQL> conn / as sysdba

Connected

SQL> SELECT STATUS''FILENAME''BYTES FROM V$BLOCK_CHANGE_TRACKING
SQL> /

STATUS''FILENAME''BYTES
--------------------------------------------------------------------------------
DISABLED

2. Then enable Change Tracking.

SQL> alter database enable block change tracking using file '/usr/local/oracle/testdw0/rman_change_track.dbf';

Database altered

The Script I used to take an Incrementally Updated Backup is rman_incr_backup.sh below. This is a sript which uses non-default RMAN location as I explicitly indication where the backups should be stored on disk. This version doesn't use FRA (Flash Recovery Area) either.

# ########################################
# !/bin/bash
# Unix controls
trap cleanup 1 2 3 15
cleanup()
{
echo "Caught CTRL-C Signal ... exiting script."
exit 1
}
# Oracle Variables
export ORACLE_SID=testdw0
export ORACLE_BASE=/usr/local/oracle
export ORACLE_HOME=/usr/local/oracle/product/10.2.0.1
export PATH=$PATH:${ORACLE_HOME}/bin
# RMAN INCREMENTALLY UPDATED BACKUPS (Window of 24 hours)
rman target=/ << EOF
CONFIGURE CONTROLFILE AUTOBACKUP ON;
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '/usr/local/oracle/backups/rman_bkps/dwbacks/atbckp_cntrlfile_testdw0%F';
run {
ALLOCATE CHANNEL RMAN_BACK_CH01 TYPE DISK FORMAT '/usr/local/oracle/backups/rman_bkps/dwbacks/databasefiles_%d_%u_%s_%T';
CROSSCHECK BACKUP;
RECOVER COPY OF DATABASE with TAG 'testdw0_incr_update';
BACKUP INCREMENTAL LEVEL 1 FOR RECOVER OF COPY with TAG 'testdw0_incr_update' DATABASE;
sql 'ALTER SYSTEM ARCHIVE LOG CURRENT';
BACKUP as compressed backupset ARCHIVELOG ALL format '/usr/local/oracle/backups/rman_bkps/dwbacks/archivelogs_%d_%u_%s_%T' DELETE INPUT;
CROSSCHECK BACKUP;
DELETE NOPROMPT OBSOLETE;
DELETE NOPROMPT EXPIRED BACKUP;
RELEASE CHANNEL RMAN_BACK_CH01;
}
EXIT;
EOF
#########################################

RMAN Recovery

In this post I will try to explain how a database recovery is done on a New Host using the latest RMAN full Hot Backup.

The recovery is done on a New Host with the same directory structure and the same database name and DBID. Lets assume that you have used the Rman Hot Backup Script provided in this blog, which takes backups of the database using a retention policy with a recovery window of 2 days. The full script is here:

http://www.databasesystems.info/2008/02/rman-hot-backup-script.html

That is, it will only keep backups and archive logs necessary to recover in any point of time in the last 2 days. During recovery, RMAN will use the latest backups and archivelogs from the backup directory.


1. On the New Host connect to RMAN after you set ORACLE_SID to newSID

oracle@NEWHOST:~ . oraenv
ORACLE_SID = [testdw0] ? testdb0



oracle@NEWHOST:~$ rman target /

Recovery Manager: Release 10.2.0.1.0 - Production on Wed Jun 20 15:00:13 2007
Copyright (c) 1982, 2005, Oracle. All rights reserved.
connected to target database (not started)

RMAN>

2. Set the DBID to the DBID of the database you want to restore and recover, you can find the DBID in backup logs or in the autobackup controlfile name.

RMAN> set DBID 347812949

executing command: SET DBID

RMAN>

3. Start the database with nomount. It will fail to startup properly with LRM-00109 as below, because you don't have init.ora file yet for your instance to start.

RMAN> startup force nomount;

startup failed: ORA-01078: failure in processing system parameters
LRM-00109: could not open parameter file '/usr/local/oracle/product/10.2.0.1/dbs/inittestdb0.ora'

starting Oracle instance without parameter file for retrival of spfile

Oracle instance started
Total System Global Area 159383552 bytes
Fixed Size 1218268 bytes
Variable Size 54528292 bytes
Database Buffers 100663296 bytes
Redo Buffers 2973696 bytes


RMAN>

4. Indicate the location of your controlfile autobackups.

As this location contains the controlfile which has built into it your init.ora as well.

RMAN> set controlfile autobackup format for device type disk to '/usr/local/oracle/backups/rman_cntrl/autobackup_control_file%F';

executing command: SET CONTROLFILE AUTOBACKUP FORMAT
using target database control file instead of recovery catalog

RMAN>

5. Restore the init.ora file.

It is perfectly valid at this point aflter you restore the init.ora to edit it to suit your needs. Change directory locations for controlfiles etc...

RMAN> run
{
allocate channel c1 type disk;
restore spfile to pfile '/usr/local/oracle/product/10.2.0.1/dbs/inittestdb0.ora' from autobackup;
shutdown abort;
};

allocated channel: c1
channel c1: sid=36 devtype=DISK

Starting restore at 20-JUN-07

channel c1: looking for autobackup on day: 20070620
channel c1: autobackup found: /usr/local/oracle/backups/rman_cntrl/autobackup_control_filec-347812949-20070620-02
channel c1: SPFILE restore from autobackup complete
Finished restore at 20-JUN-07

Oracle instance shut down

RMAN>

6. Restart the instance with the proper init.ora file you restoed in step 5.

RMAN> startup force nomount;

Oracle instance started

Total System Global Area 608174080 bytes

Fixed Size 1220844 bytes
Variable Size 192941844 bytes
Database Buffers 411041792 bytes
Redo Buffers 2969600 bytes

RMAN>

7. Restore the controlfile

First indicate where is the controlfile you want to use.

RMAN> set controlfile autobackup format for device type disk to '/usr/local/oracle/backups/rman_cntrl/autobackup_control_file%F';

executing command: SET CONTROLFILE AUTOBACKUP FORMAT

Now restore the controlfile.

RMAN> RESTORE CONTROLFILE FROM AUTOBACKUP;

Starting restore at 20-JUN-07
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=156 devtype=DISK

channel ORA_DISK_1: looking for autobackup on day: 20070620
channel ORA_DISK_1: autobackup found: /usr/local/oracle/backups/rman_cntrl/autobackup_control_filec-347812949-20070620-02
channel ORA_DISK_1: control file restore from autobackup complete
output filename=/usr/local/oracle/testdb0/control01.ctl
output filename=/usr/local/oracle/testdb0/control02.ctl
output filename=/usr/local/oracle/testdb0/control03.ctl
Finished restore at 20-JUN-07

released channel: ORA_DISK_1

RMAN>

8. Now you have a controlfile, mount the database

RMAN> ALTER DATABASE MOUNT;

database mounted

RMAN>

9. Indicate RMAN where your hotbackups are

This is where you tell RMAN where you backups are. Try changing backup files directory and re-run this command with the appropriate location and you will see that it will 'find' the files and update, the control file or catalog, accordingly.

RMAN> CATALOG START WITH '/usr/local/oracle/backups/rman_bkps/hot_backups/';

searching for all files that match the pattern /usr/local/oracle/backups/rman_bkps/hot_backups/
no files found to be unknown to the database

RMAN>

10. Restore the database

RMAN> restore database;

Starting restore at 20-JUN-07
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=156 devtype=DISK

channel ORA_DISK_1: starting datafile backupset restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
restoring datafile 00001 to /usr/local/oracle/testdb0/system01.dbf
restoring datafile 00002 to /usr/local/oracle/testdb0/undotbs01.dbf
restoring datafile 00003 to /usr/local/oracle/testdb0/sysaux01.dbf
restoring datafile 00004 to /usr/local/oracle/testdb0/users01.dbf
restoring datafile 00005 to /usr/local/oracle/testdb0/ts1.dbf
restoring datafile 00006 to /usr/local/oracle/testdb0/apex.dbf
restoring datafile 00007 to /usr/local/oracle/testdb0/FLOW_1.dbf
channel ORA_DISK_1: reading from backup piece /usr/local/oracle/backups/rman_bkps/hot_backups/databasefiles_TESTDB0_s3iko9hm_899_20070620
channel ORA_DISK_1: restored backup piece 1
piece handle=/usr/local/oracle/backups/rman_bkps/hot_backups/databasefiles_TESTDB0_s3iko9hm_899_20070620 tag=TAG20070620T104510
channel ORA_DISK_1: restore complete, elapsed time: 00:03:16
Finished restore at 20-JUN-07

RMAN>

11. Recover the database, will automatically apply archivelogs and will stop (will fail like below on the last log)

This is normal as Oracle will always try to roll-forward the database to the latest log, will always ask for the latest archivelogs and in this will stop on the last available one. At this point we will assume that we are doing CANCEL based recovery and open the database with RESETLOGS option. You could avoid this error by first finding out which archivelog sequence is the last one and the recovering up to that point. Since RMAN does NOT do CANCEL BASED RECOVERY. Nevertheless here is what happens if you just say RECOVER DATABASE at this point

RMAN> recover database;

Starting recover at 20-JUN-07
using channel ORA_DISK_1

starting media recovery

channel ORA_DISK_1: starting archive log restore to default destination
channel ORA_DISK_1: restoring archive log
archive log thread=1 sequence=333
channel ORA_DISK_1: restoring archive log
archive log thread=1 sequence=334
channel ORA_DISK_1: reading from backup piece /usr/local/oracle/backups/rman_bkps/hot_backups/archivelogs_TESTDB0_s5iko9p1_901_20070620
channel ORA_DISK_1: restored backup piece 1
piece handle=/usr/local/oracle/backups/rman_bkps/hot_backups/archivelogs_TESTDB0_s5iko9p1_901_20070620 tag=TAG20070620T104905
channel ORA_DISK_1: restore complete, elapsed time: 00:00:25
archive log filename=/usr/local/oracle/arch/testdb0_1_333_614535884.arc thread=1 sequence=333
archive log filename=/usr/local/oracle/arch/testdb0_1_334_614535884.arc thread=1 sequence=334
unable to find archive log
archive log thread=1 sequence=335
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 06/20/2007 15:22:07
RMAN-06054: media recovery requesting unknown log: thread 1 seq 335 lowscn 1793924713

RMAN>

You could have avoided seing this ugly error message at the end of your recovery, if you only knew until which archivelog sequence number (usually last sequence number avaliable) you want to recover to. You can obtain this last sequence number by issuging the command:

RMAN> list backup of archivelog from time='sysdate-7';

This would have shown you the last available sequence number from your archivelog backups and then you would more gracefully recover to that sequence number like this:

RMAN> recover database until sequence=;

Then next, you would open the database as in step 12 below.


12. Then you have to open the database with RESETLOGS option


You have to since you are doing a cancel based recovery from a full hot backups with no online redo logs. Redo logs are not present and this step creates them and resets them to 0.

RMAN> alter database open resetlogs;

database opened

RMAN>

You are done! You have now the database fully operational on the new host from an RMAN full hot backup.

LAST_VALUE analytic functions

Oracle analytic functions are great! There is a few of them which I like particularly and would like to post here. I always need to check tables for the last entry of a record.

LAST_VALUE

The LAST_VALUE analytic function returns the last_value of an ordered set of values. Think of a table with data like this:

SQL> select * from tab1;

        ID     ENDEKS ISIM
---------- ---------- ----------
         1       1001 AA
         1       1002 CV
         1       1003 FC
         2       1001 AA
         3       1001 KK
         3       1000 LL
         2       1002 ZZ
         2       1010 MM

Suppose, all you wanted to see is the last entry for each ID at any point in time, then an Oracle Analytics SQL Query like this would do.
SQL> select x.id,max(endeks), isim from
(
SELECT ID,
last_value(endeks) over(PARTITION BY ID order by endeks asc) endeks,
last_value(isim) over(PARTITION BY Id ) isim
FROM tab1
order by 1 , 2
) X
group by x.id, isim
order by 1
/

     ID MAX(ENDEKS) ISIM
---------- ----------- ----------
         1        1003 FC
         2        1010 MM
         3        1001 KK

Same answer with MAX and Oracle Analytics

It is possible to get the same answer with Oracle Analytics and the MAX function. If your table is very big, use this one as it is much more efficient as it goes through your table only once. Use Autotrace for both statements to see the difference.
SQL> 1 select id, endeks, isim from
(
select a.*, max(endeks) over (partition by id) last_message
from tab1 a
)
where endeks = last_message
/

        ID     ENDEKS ISIM
---------- ---------- ----------
         1       1003 FC
         2       1010 MM
         3       1001 KK

Configuring Oracle Standby Database on Ubuntu Linux

In this post I will write the steps I took to configure a Physical Standby Database on two Ubuntu Hosts versions Breezy Badger or Dapper Drake and running Oracle 10g R2. For all this to work


following assumptions must hold true:

Both hosta and hostb must have the same operating system and version.

The directory structure for datafiles on both hosts must be the same.

1. Create the Primary Database.

I have created a small database on hosta called testdb1 to use as the Primary Database using dbca. Then I took a cold backup of this database.

2. Move backups of Primary Database to Standby Database on hostb.

Next I moved the backup datafiles to hostb in their corresponding same locations with scp like this:

scp /u10/oradata/testdb1/* oracle@hostb:/u10/oradata/testdb1/

3. Create the Standby Database control file.

Then I went back to hosta and started up testdb1 Primary Database, and created a special control file. It is important to create the controlfile after you shutdown the Primary Database cleanly and take the backup and not before. This controlfile will be used on hostb to startup the Standby Database.
Log in to Primary Database on hosta as SYSDBA and run the command:

SQL> alter database create standby controlfile as ‘/tmp/sbycontrol.ctl’

NOTE: This file is binary. Use a different filename than the controlfiles of the Primary Database and watch out not to overwrite a current Primary Database control files.

Then move the created controlfile to hostb. You can choose any location on hostb. Later, you will manually edit the initialization parameter file of the Standby Database to indicate the chosen location.


SQL> !scp /tmp/sbycontrol.ctl oracle@hostb:/u10/oradata/testdb1/

4. Create the initialization parameter file for the Standby Database.

Create a pfile from the spfile of the Primary Database to be used later as the intialization parameter file to startup the Standby Database. The initialization parameter file you will create from the Primary Database will be manually modified to introduce Standby Database parameters later.
Log in to Primary Database on hosta as SYSDBA and run the command:

SQL> create pfile=’/tmp/inittestdb1.ora’ from spfile

Then copy the file to $ORACLE_HOME/dbs on hostb, to the default location. You will edit the file later on hostb.

SQL> !scp /tmp/inittestdb1.ora oracle@hostb:$ORACLE_HOME/dbs

5. Set up networking configuration between Primary and Standby Databases.

You have to make sure that both databases will be able to communicate using their own service names pointing to each other. Create service names on both hosta and hostb which will connect to each other and ship archived logs.
Edit the tnsnames.ora file on hosta and hostb and add the service name TO_STANDBY to both.

in tnsnames.ora on hosta add:

TO_STANDBY =(DESCRIPTION =(ADDRESS_LIST =(ADDRESS = (PROTOCOL = TCP)(HOST = hosta)(PORT = 1521)))(CONNECT_DATA =(SERVICE_NAME = testdb1)))

in tnsnames.ora on hostb add:

TO_STANDBY =(DESCRIPTION =(ADDRESS_LIST =(ADDRESS = (PROTOCOL = TCP)(HOST = hostb)(PORT = 1521)))(CONNECT_DATA =(SERVICE_NAME = testdb1)))


You must also create the password file on standby as well as primary database to enable logins from each other. The password for these must be the same.
on hosta and hostb do

$orapwd file=$ORACLE_HOME/dbs/orapw$ORACLE_SID password=yourpassword

6. Edit the pfile created for Standby Database

In step 4 above an initialization parameter file was created from the Primary Database. Now is time to make that file the initialization parameter of the Standby Database and add the parameters necessary to startup the Standby Database in Standby mode.

The file created earlier should be in $ORACLE_HOME/dbs directory like:
/usr/local/oracle/product/10.2.0.1/dbs/inittestdb1.ora

According to the method of synchronization (redo transport) you choose between the Primary Database and the Standby Database certain parameters will be set in this file.

For my installation I used Standby Logs on the Standby Database and Archived Log Transmission.

I edited the Standby Database initialization parameter file as below, where bold is what I added.

…
*.compatible=’10.2.0.1.0′
*.control_files=’/u10/oradata/testdb1/sbycontrol.ctl’…
*.log_archive_dest_1=’LOCATION=/u01/arch’
*.log_archive_dest_2=’LOCATION=/u11/arch’
*.standby_archive_dest=’LOCATION=/u00/sbylogs’
*.standby_file_management=’AUTO’
*.remote_archive_enable=’TRUE’
*.log_archive_format=’testdb1_%t_%s_%r.arc”
*.undo_tablespace=’UNDOTBS1′
*.user_dump_dest=’/usr/local/oracle/admin/testdb1/udump’
...

Once you edited the Standby Database initialization parameter file next is to make an spfile out of it and startup the Standby Database.

7. Create spfile for Standby Database and start it up

Log in to Standby Database on hostb as SYSDBA and run the command:

SQL> create spfile from pfile=’/usr/local/oracle/product/10.2.0.1/dbs/inittestdb1.ora’;

Next startup the Standby Database on hostb with NOMOUNT

SQL> startup nomount
ORACLE instance started.
Total System Global Area 167772160 bytes
Fixed Size 1218316 bytes
Variable Size 62916852 bytes
Database Buffers 100663296 bytes
Redo Buffers 2973696 bytes

And then MOUNT the Standby Database as standby!

SQL> alter database mount standby database;
Database altered.

Now the Standby Database has started, it has got a control-file, but hasn’t got standby logs where redo data will be applied as redo comes from the Primary Database and is applied to the standby database via its archivelogs. Next is to add these standby logs on hostb to the standby database testdb1.
NOTE: If you want to configure fail-over between the two host where each can assume the role of Primary Database / Standby Database then is a good idea to add these standby logs to the Primary Database as well. I don’t explain how to do that here. In this post I have only explained how to configure “A physical standby database for a primary database” Is best if you read the Oracle documentation for that. There is a good explanation of that here:
http://download-uk.oracle.com/docs/cd/B19306_01/server.102/b14239/create_ps.htm

8. Add Standby Logs to your Standby Database

I added 11 logs using the following statement.
Log in to Standby Database on hostb as SYSDBA and run the command:

SQL> alter database add standby logfile ‘/u00/sbylogs/sbylog07.log’ size 5M;
…
SQL> alter database add standby logfile ‘/u00/sbylogs/sbylog011.log’ size 5M;

It is important and there is a good argument to chose twice the number of primary database redo logs as standby logs on the standby database, for reasons of archiving contention. That is during log shipping you don’t want the system to get bottlenecked, due to redo log switches and archiving not being able to cope with them.

9. On hosta on the Primary Database start shipping logs

Log in to Primary Database on hosta as SYSDBA and run the command:
alter system set log_archive_dest_3=’SERVICE=TO_STANDBY ARCH’;
alter system set log_archive_dest_state_3=enable scope=both;
This will start the process of shipping the archivelogs to the standby database from a brand new log archive destination which is pointing to the standby database, check the service name used TO_STANDBY
NOTE: At this point is best to have windows open to both alert.log logs of standby and primary and see what is hapenning and to spot errors

10. Put the Standby Database in recovery mode (Last Step)

Last step, is to put the Standby database in recovery mode. That is the standby database in the “Physical” configuration stays mounted in recovery mode. Is not open for querying.

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;

That is it! You have a standby database on hostb!

If you want to test and see your system working, for examble by changing something in the primary database on hosta and seeing the change being applied to the standby database on hostb you will need to open the standby database. Steps on how to stop the standby database from receiving Archivelogs from the primary database, and how to put it back to the recovery mode, are below.
Switch from log shipping mode to read only standby database
Cancel Redo Apply:

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;

Open the database for read-only access:

SQL> ALTER DATABASE OPEN;

Now you can query the standby database and see the changes you made in primary database. You can NOT do DML on the standby database! That is this is a one way synch from primary to standby.

Change the standby database from being open for read-only access to performing Redo Apply: Terminate all active user sessions on the standby database, you my need to kill the sessions except yours.
Restart Redo Apply. To start Redo Apply, issue the following statement:

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;

Now the standby database is in recovery mode/ standby mode and is receiving Archivelogs from the primary database.

In future post I will write on how to create a physical standby database using RMAN and how to configure Oracle Data Guard Broker for automatic switchover/failover operations and High Availability solutions.



References:

http://download-uk.oracle.com/docs/cd/B19306_01/server.102/b14239/create_ps.htm
http://www.adp-gmbh.ch/ora/data_guard/create_physical_standby_db.html

Using Data Guard to do fail-over and other cool features here:
http://download-uk.oracle.com/docs/cd/B19306_01/server.102/b14230/cli.htm#i1005573

Oracle Table Compression

Oracle employs a very nice compression algorithm on database block level which is quite efficient and can yield threefold compression ratio in most cases.

The table compression is highly recommended for Data Warehouse environments where the operations tend to be read heavy and batch. Table compression is not recommended for OLTP environments or on tables with high OLTP like operations because of the cost associated with DML operations on compressed tables. This is true with Oracle 10g. Oracle 11g has gone wild, and offers OLTP compression as well!

I have found this quite nice short explenation in Oracle Documentation about what Oracle Table compression is:

"...Oracle table compression feature compresses data by eliminating duplicate values in a database block. Compressed data stored in a database block (also known as disk page) is self-contained. That is, all the information needed to re-create the uncompressed data in a block is available within that block. Duplicate values in all the rows and columns in a block are stored once at the beginning of the block, in what is called a symbol table for that block. All occurrences of such values are replaced with a short reference to the symbol table..."

The benefits associated with Table Compression are:

• Save space
• Faster backups
• Query Performance

1. Find out the candidate table to compress

Usually NUM_ROWS will reveal the table in need for compression. In Data Warehouses these are usually fact tables. The fact tables tend to have lots of redundant data in the columns such as keys, data and status flag data. The compression algorithm uses a symbolic table structure to record this redundancy numerically rather than physically and repeatedly a the table level. That is compression on the 'Relational Table' level rather than byte level.

Something like this:

Table A Uncompressed

a1	a2
x	z
x	z
x	m
x	z
y	m

Table A When compressed becomes

a1	a1 sl	a2	a2 sl
x	4	z	3
y	1	m	2

My worked example:

As example we will use the X table, the largest table which I will partition and compress all its partitions except the current partition. The reason we don't compress the current partition is because in the current month partition there will be load of ETL activity (DML) and we do not want to introduce high CPU usage because of compression. Compression is no good when there is lots of DML.
The size of the table before compression is 6GB:

SQL> select segment_name, sum(bytes)/1024/1024 MB from user_Segments where segment_name like 'X'
group by segment_name
SQL> /

SEGMENT_NAME MB
------------------------------ ----------
X 6179

2. List the partitions of the candidate table and decide which to compress

You can list the partition names for the tables you are interested to compress like this:

SQL> set pages 49999
SQL> l

1* select table_name, partition_name, compression from user_tab_partitions where table_name ='X'

SQL> /TABLE_NAME PARTITION_NAME COMPRESS

------------------------------ ------------------------------ --------

...

X P0201 DISABLED
X P0202 DISABLED

...
Then you can write an SQL query to build the command which will move and compress the partitions. Just by altering the property of the table to COMPRESS => ENABLED doesn't compress existing rows in the table. It will compress only the new rows which will be added to the table. To save space we must compress what is there and leave only the current partition and future partitions uncompressed. You must remember to NOT compress the current partition and the future partitions, as the script below will generate commands to compress all partitions. To find out the current partition look at the PARTITION_POSITION in USER_TAB_PARTITIONS, the highest positioned partition is the current partition. You can also order by PARTITION_POSITION as below and don't do the first line.
Generate PARTITION COMPRESS commands with the following script.

SQL> select 'ALTER TABLE 'table_name' MOVE PARTITION 'partition_name ' TABLESPACE NEW_TABLESPACE COMPRESS NOLOGGING PARALLEL; ' from user_tab_partitions where table_name ='X'
order by partition_position desc;

'ALTERTABLE'TABLE_NAME'MOVEPARTITION'PARTITION_NAME'TABLESPACENEW_TABLESPACECOMPRESSNOLOGGINGPARALLEL;'

---------------------------------------------------------------------------------------------------
ALTER TABLE X MOVE PARTITION P0207 TABLESPACE NEW_TABLESPACE COMPRESS NOLOGGING PARALLEL;
ALTER TABLE X MOVE PARTITION P0107 TABLESPACE NEW_TABLESPACE COMPRESS NOLOGGING PARALLEL;

WARNING : It is very important to choose to compress the tables in NEW tablespace as compressing them alone doesn't reorganize the extents in the datafiles in a way for them to be able to be re-sized and shrunk later on.

3. If there are bitmap indexes on the candidate table you must first mark them unusable

WARNING : To be able to apply the commands below the table in question must not have bitmap indexes , if it does you must mark them as unusable otherwise you get ORA-14646 err.

ORA-14646: Specified alter table operation involving compression cannot be performed
in the presence of usable bitmap indexes

See here for more info :http://download-uk.oracle.com/docs/cd/B19306_01/server.102/b14223/parpart.htm#i1006787

You will have to rebuild the indexes after you have finished with compressing the partitions.

4. Run the compression commands

For each partition you wish to compress you are now ready to run the commands like :

ALTER TABLE X MOVE PARTITION P0207 TABLESPACE NEW_TABLESPACE COMPRESS NOLOGGING PARALLEL;
ALTER TABLE X MOVE PARTITION P0107 TABLESPACE NEW_TABLESPACE COMPRESS NOLOGGING PARALLEL;

You can choose to use NOLOGGING for this operation as it generates lots of redo and will affect your archive logging and PARALLEL to make it faster.

After compressing all partitions on the X table the savings realised are as follows:

A 41% saving!

-- Before Compression

SEGMENT_NAME MB
------------------------------ ----------
X 6179

-- After Compression

SEGMENT_NAME MB
------------------------------ ----------
X 3670

5. You must rebuild the BITMAP INDEXES marked UNUSABLE at the previous step 4.


You will have to rebuild all bitmap , and other if any, indexes marked unusable at the previous steps, to do it quickly you ca use NOLOGGING and PARALLEL. Similarly write an SQL statement querying the USER_IND_PARTITIONS table and generate statement like the following for all indexes.

ALTER INDEX my_bitmap_index REBUILD PARTITION P1001 NOLOGGING;

ALTER INDEX my_bitmap_index REBUILD PARTITION P1002 NOLOGGING;

...
This is a one-off operation you won't do it again for the table, ie when you add a new partition.

6. You must rebuild ALL LOCAL INDEXES in the partitioned tables as their state has now become UNUSABLE

After the compression of the table partitions all local indexes associate with the table become UNUSABLE. As the oracle documentation categorically indicates:

If you use the ALTER TABLE ... MOVE statement, the local indexes for a partition become unusable. You have to rebuild them afterwards. Otherwise you will get errors in partition maintenance operations like this:

*

ERROR at line 1:

ORA-01502: index 'my_schema.my_X_table_index' or partition of such index is in unusable

state

You can do that by writing something like this and change the table_name=X for each table.

SQL> select 'alter table X modify partition ' partition_name ' rebuild unusable local indexes;' from user_tab_partitions where table_name='X'

NOTE: You will also have to rebuild indexes whenever you COMPRESS/NOCOMPRESS any partitions. That is, when you use any of these two operations, data moves, and indexes break and have to be rebuild.

7. Check to see the space you have gained.

After moving the table partition segments to their new tablespace and rebuilding necessary indexes you ought to gain some space. You will realize that the space which the objects before compression has decreased. You will require less space to store them, when compressed, in their new tablespace. After you move everything from the old tablespace into the new and you are sure the old tablespace is empty you can then drop the old tablespace and its datafiles.

Well the only approach to 'claim' space back after a table compression operation seems to be the use of ALTER TABLE ... MOVE TABLESPACE 'YOURNEWTABLESPACE' on a 'new' tablespace. Although we can claim physical space back from the database only by resizing datafiles and we could have 'moved' the tables withing their 'own' tablespace, it is not recommended.

Moving the tables within the same tablespace is not a good idea, because you are never sure if the reorganization of the table segments will actually move the HWM of the datafile down so that you can resize the datafile. (see below asktom.oracle.com links for more info)

There might be 'that' extent still hanging at the end of the tablespace which will prevent you from resizing the datafiles. Although you can fish that 'hanging at the end' extent out and reorganize the segment you are better off by moving all to a new tablespace.

Use DBA_SEGMENTS before you decide to drop a tablespace to see if any segments are left in it. Alternativaly you can choose to resize the datafiles to very small files.

I am also reading lately that in Oracle 11g you can get Advanced Table Compression in OLTP environments as well! It is an option which you have to puchase separately in Enterprise Edition though.

References:

Forum Questions to Asktom :

http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:7873209745771#240368600346159666

http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:54178027703899

Oracle Docs :

http://download-uk.oracle.com/docs/cd/B19306_01/server.102/b14223/parpart.htm#sthref196