jump to navigation

12.2 Online Conversion of a Non-Partitioned Table to a Partitioned Table (A Small Plot Of Land) March 27, 2017

Posted by Richard Foote in 12c Release 2 New Features, Attribute Clustering, Clustering Factor, Online DDL, Oracle, Oracle Indexes, Partitioning.
1 comment so far

Image result for outside bowie

In my previous post, I discussed how you can now move heap tables online with Oracle Database 12.2 and how this can be very beneficial in helping to address issues with the Clustering Factor of key indexes.

A problem with this technique is that is requires the entire table to be effectively reorganised when most of the data might already be well clustered. It would be much more efficient if we could somehow only move and reorganise just the portion of a table that has poorly clustered data introduced to the table since the last reorg.

Partitioning the table appropriately would help to address this disadvantage but converting a non-partitioned table to be partitioned can be a pain. To do this online with as little complication as possible one could use the dbms_redefintion package which has improved with latter releases.

However, with Oracle Database 12.2, there is now an even easier, more flexible method of performing such a conversion.

Using the same table definition and data as from my previous post, I’m going to first create a couple of additional indexes (on the ID column and on the DATE_CREATED column) :


SQL> create unique index ziggy_id_i on ziggy(id);

Index created.

SQL> create index ziggy_date_created_i on ziggy(date_created);

Index created.

To convert a non-partitioned table to a partitioned table online, we can now use this new extension to the ALTER TABLE syntax:


SQL> alter table ziggy
2 modify partition by range (date_created)
3 (partition p1 values less than (TO_DATE('01-JAN-2015', 'DD-MON-YYYY')),
4 partition p2 values less than (TO_DATE('01-JAN-2016', 'DD-MON-YYYY')),
5 partition p3 values less than (maxvalue)) online;

Table altered.

How simple is that !! We now have a table that is range partitioned based on the DATE_CREATED column and this conversion was performed online.

We notice not only is the table now partitioned with all the indexes remaining Valid, but the index based on the partitioning key (DATE_CREATED) has also been implicitly converted to be a Local partitioned index:


SQL> select table_name, status, partitioned from dba_tables
where table_name='ZIGGY';

TABLE_NAME   STATUS   PAR
------------ -------- ---
ZIGGY        VALID    YES

SQL> select index_name, status, partitioned, num_rows
from dba_indexes where table_name='ZIGGY';

INDEX_NAME           STATUS   PAR   NUM_ROWS
-------------------- -------- --- ----------
ZIGGY_DATE_CREATED_I      N/A YES    2000000
ZIGGY_CODE_I VALID             NO    2000000
ZIGGY_ID_I VALID               NO    2000000

SQL> select index_name, partition_name, status, leaf_blocks from dba_ind_partitions
     where index_name like 'ZIGGY%';

INDEX_NAME           PARTITION_NAME  STATUS   LEAF_BLOCKS
-------------------- --------------- -------- -----------
ZIGGY_DATE_CREATED_I              P1   USABLE         865
ZIGGY_DATE_CREATED_I              P2   USABLE        1123
ZIGGY_DATE_CREATED_I              P3   USABLE        1089

SQL> select index_name, partitioning_type, partition_count, locality
from dba_part_indexes where table_name='ZIGGY';

INDEX_NAME           PARTITION PARTITION_COUNT LOCALI
-------------------- --------- --------------- ------
ZIGGY_DATE_CREATED_I     RANGE               3 LOCAL

As part of the table conversion syntax, we have the option to also update all the associated indexes and partition them in any manner we may want. For example:


SQL> alter table ziggy
2 modify partition by range (date_created)
3 (partition p1 values less than (TO_DATE('01-JAN-2015', 'DD-MON-YYYY')),
4 partition p2 values less than (TO_DATE('01-JAN-2016', 'DD-MON-YYYY')),
5 partition p3 values less than (maxvalue)) online
6 update indexes
7 (ziggy_code_i local,
8 ziggy_id_i global partition by range (id)
9 (partition ip1 values less than (maxvalue)));

Table altered.

In this example, not only are we converting the non-partitioned table to be partitioned, but we’re also explicitly converting the index on the CODE column to be a Locally partitioned index and the index on the ID column to be Globally partitioned in its own manner.

If we look at the definition of these indexes, we see that they also have all been converted to partitioned indexes online along with the table:


SQL> select table_name, status, partitioned from dba_tables
where table_name='ZIGGY';

TABLE_NAME   STATUS   PAR
------------ -------- ---
ZIGGY           VALID YES

SQL> select index_name, status, partitioned from dba_indexes
where table_name = 'ZIGGY';

INDEX_NAME           STATUS   PAR
-------------------- -------- ---
ZIGGY_CODE_I              N/A YES
ZIGGY_ID_I                N/A YES
ZIGGY_DATE_CREATED_I      N/A YES

SQL> select index_name, partitioning_type, partition_count, locality
from dba_part_indexes where table_name='ZIGGY';

INDEX_NAME           PARTITION PARTITION_COUNT LOCALI
-------------------- --------- --------------- ------
ZIGGY_CODE_I             RANGE               3 LOCAL
ZIGGY_ID_I               RANGE               1 GLOBAL
ZIGGY_DATE_CREATED_I     RANGE               3 LOCAL

If we look at the Clustering Factor of the important CODE column index, we see that all partitions have an excellent Clustering Factor as all partitions have just been created.


SQL> select partition_name, num_rows, clustering_factor from dba_ind_partitions
where index_name='ZIGGY_CODE_I';

PARTITION_NAME         NUM_ROWS CLUSTERING_FACTOR
-------------------- ---------- -----------------
P1                       490000              2275
P2                       730000              3388
P3                       780000              3620

However, if we now add new rows to the table as would occur with a real application, the data from the “current” partition results in the Clustering Factor “eroding” over time for this partition.


SQL> insert into ziggy select 2000000+rownum, mod(rownum,100), sysdate, 'DAVID BOWIE'
from dual connect by level <= 500000; 500000 rows created. SQL> commit;

Commit complete.

SQL> exec dbms_stats.gather_index_stats(ownname=>null,indname=>'ZIGGY_CODE_I');

PL/SQL procedure successfully completed.

SQL> select partition_name, num_rows, clustering_factor from dba_ind_partitions
     where index_name='ZIGGY_CODE_I';

PARTITION_NAME         NUM_ROWS CLUSTERING_FACTOR
-------------------- ---------- -----------------
P1                       490000              2275
P2                       730000              3388
P3                      1280000            238505

As discussed previously, the Clustering Attribute has no effect with standard DML operations. Therefore, the efficiency of the CODE index reduces over time in the partition where new data is being introduced. The Clustering Factor has now substantially increased from 3620 to 238505. Note for all the other partitions where there are no modifications to the data, the Clustering Factor remains excellent.

Having the table/index partitioned means we can therefore periodically reorg just the problematic partition:


SQL> alter table ziggy move partition p3 update indexes online;

Table altered.

SQL> select partition_name, num_rows, clustering_factor from dba_ind_partitions
     where index_name='ZIGGY_CODE_I';

PARTITION_NAME         NUM_ROWS CLUSTERING_FACTOR
-------------------- ---------- -----------------
P1                       490000              2275
P2                       730000              3388
P3                      1280000              5978

The Clustering Factor for this partition has now reduced substantially from 238505 to just 5978.

For those of you with the Partitioning database option, the ability in 12.2 to now so easily convert a non-partitioned table to be partitioned, along with its associated indexes is just brilliant 🙂

Let’s Talk Database: New Dates Released (You Can’t Talk) March 16, 2017

Posted by Richard Foote in Let's Talk Database, Oracle Indexes.
3 comments

Oracle-Database

Good news for all you Oracle Database folk in ANZ. I’ll be running a new series of “Let’s Talk Database” sessions at a venue near you in the coming months. I’ll be discussing two interesting topics, “Introduction to the Oracle Cloud for DBAs” and “Introduction to Oracle Database 12c Release 2“. The dates for this series and registration links as follows:

Agenda

8:30 – 9:00am Registration and coffee
9:00 – 10:00am Introduction to Oracle Cloud for DBAs
10:00 – 10:15am Break
10:15 – 11:15am Introduction to Oracle Database 12c Release 2–Part I
11.15 – 11:30am Break
11:30 – 12:30pm Introduction to Oracle Database 12c Release 2–Part II
12:30 – 1:30pm Lunch, Networking and informal Q&A

Session Abstracts

Introduction to the Oracle Cloud for DBAs

The Oracle Cloud has many key features and benefits that would be of much interest to Oracle DBAs including increased flexibility and elasticity, better security and compliance, reduced complexity with regard to maintenance and patching, improved access to database features and faster database deployments. This session will introduce the various options and capabilities available when deploying Oracle Databases within the Oracle Public Cloud, such as database services and deployment options, database option bundling, backup/recovery and DR services, hybrid manageability features and database accessing options. Oracle Exadata Express Cloud Service will be used as an example of how to quickly deploy and access an Oracle Database in the cloud. We will also look at some options with regard to a Hybrid Cloud model, including deploying Cloud based infrastructure within your own data centre. Finally, we introduce the Oracle Cloud data centre which has recently been expanded in Sydney, Australia.

Introduction to Oracle Database 12c Release 2

Oracle Database 12c Release 2 continues Oracle’s innovation with its leading database with the introduction of many new exciting features and enhancements. This session will discuss in some detail many of these new 12.2 capabilities that makes the Oracle Database the cloud database of choice, including improvements with the Database In-Memory option, new indexing compression and tracking features, new online maintenance operations such as online tablespace encryption, online table move, online table/index partition conversion, new pluggable database capabilities such as hot cloning and application containers, Oracle Sharding, Active Data Guard and RAC improvements, Big Data and Analytics innovations as well as several new developer features such as JSON support improvements, SQL enhancements and case insensitive support.

So lots to talk about and lots of opportunity to ask questions and network with your fellow Oracle professionals.

As usual, places are unfortunately limited so contact your location and enroll ASAP to avoid disappointment.

Hope to see many of you at these upcoming events 🙂

12.2 Moving Tables Online (Real Cool World) March 15, 2017

Posted by Richard Foote in Oracle Indexes.
2 comments

real cool world

One of the really cool new features introduced in Oracle Database 12c Release 2 is the capability to MOVE tables ONLINE, while maintaining all the associated index structures. This was possible for table partitions in Oracle Database 12c Release 1 but only for Indexed Organized Tables before then.

Rather than show a demo that just moves a table, thought I might also provide a classic example of when moving a table might be extremely beneficial.

I’ll start by creating a table and specifically populate a CODE column with 100 distinct values that are distributed throughout the entire table structure:


SQL> create table ziggy (id number, code number, date_created date, name varchar2(42));

Table created.

SQL> insert into ziggy select rownum, mod(rownum,100), sysdate – mod(rownum,1000), DAVID BOWIE’ from dual connect by level <= 2000000; 2000000 rows created. SQL> commit;

Commit complete.

SQL> exec dbms_stats.gather_table_stats(ownname=>user, tabname=>'ZIGGY', estimate_percent=>null, method_opt=>'FOR ALL COLUMNS SIZE 1');

PL/SQL procedure successfully completed.

If I create an index on the CODE column, we’ll see the index has a terrible Clustering Factor (a massive 923,900), as the values basically appear throughout the entire table in each of the table blocks:


SQL> create index ziggy_code_i on ziggy(code);

Index created.

SQL> select index_name, clustering_factor, num_rows from user_indexes
where index_name='ZIGGY_CODE_I';

INDEX_NAME           CLUSTERING_FACTOR   NUM_ROWS
-------------------- ----------------- ----------
ZIGGY_CODE_I                    923900    2000000

If we now run a query that returns rows from just one CODE value (i.e. just 1% of the table):


SQL> set arraysize 5000
SQL> select * from ziggy where code = 42;

20000 rows selected.

Execution Plan
-----------------------------------------------------------------------------------
| Id | Operation                 | Name  | Rows  | Bytes | Cost (%CPU) | Time     |
-----------------------------------------------------------------------------------
| 0  | SELECT STATEMENT          |       | 20000 |  546K |    2750 (1) | 00:00:01 |
|* 1 | TABLE ACCESS STORAGE FULL | ZIGGY | 20000 |  546K |    2750 (1) | 00:00:01 |
-----------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

1 - storage("CODE"=42)
filter("CODE"=42)

Statistics
----------------------------------------------------------
     0 recursive calls
     0 db block gets
  9354 consistent gets
     0 physical reads
     0 redo size
367165 bytes sent via SQL*Net to client
   533 bytes received via SQL*Net from client
     5 SQL*Net roundtrips to/from client
     0 sorts (memory)
     0 sorts (disk)
 20000 rows processed

We see the Cost Based Optimizer (CBO) ignores the index and opts for a Full Table Scan. With such a terrible Clustering Factor and with each block in the table having to be accessed, the CBO is making the correct decision here. The index is effectively useless, even though we’re only interested in just 1% of the table. At 9,354 consistent gets, it’s the best we can do.

If this query was important to us and executed frequently, we might want to look at improving the Clustering Factor and hence the efficiency of this index. Unlike rumors to the contrary (which unfortunately still exist, believe), the key here is not to rebuild the index, but to reorganise the table such that data in the table is sorted/clustered in CODE order.

Oracle database 12.1.0.2 introduced the concept of the table Clustering Attribute, by which the table after a reorg or bulk load will store the data based on this Clustering Attribute (as I’ve discussed previously). For example:


SQL> alter table ziggy add clustering by linear order(code) without materialized zonemap;

Table altered.

This now ensures that the data within the table will be in CODE order after a table re-org, but we need to re-org the table for this to take effect. However, prior to 12.2, it wasn’t possible to subsequently MOVE the table ONLINE:


SQL> alter table ziggy move online;
alter table ziggy move online
*
ERROR at line 1:
ORA-01735: invalid ALTER TABLE option

Moving a heap table previously caused locking issues and resulted in all associated indexes beoming unusable and having to be rebuilt, further adding to the effective unavailability of the table:


SQL> alter table ziggy move;
alter table ziggy move
*
ERROR at line 1:
ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired

SQL> alter table ziggy move;

Table altered.

SQL> select index_name, leaf_blocks, status from dba_indexes where table_name = 'ziggy';

INDEX_NAME      LEAF_BLOCKS STATUS
--------------- ----------- --------
ZIGGY_CODE_I           1936 UNUSABLE

That’s now changed with Oracle Database 12c Release 2, where we can now move heap tables online while fully maintaining all associated indexes:


SQL> alter table ziggy move online;

Table altered.

SQL> select index_name, clustering_factor, num_rows, status from user_indexes
where index_name='ZIGGY_CODE_I';

INDEX_NAME      CLUSTERING_FACTOR   NUM_ROWS STATUS
--------------- ----------------- ---------- ------
ZIGGY_CODE_I                 9277    2000000 VALID

After the table re-org, we note that not only is the index still in a valid state, but because the table re-org effectively sorted the data in CODE order due to the Clustering Attribute, it has result in a much much reduced Clustering Factor (just 9,277 reduced down from 923,900).

If we now re-run our original query:


SQL> select * from ziggy where code=42;

20000 rows selected.

Execution Plan
----------------------------------------------------------------------------------------------------
| Id | Operation                           | Name         | Rows  | Bytes | Cost (%CPU) | Time     |
----------------------------------------------------------------------------------------------------
| 0  | SELECT STATEMENT                    |              | 20000 |  546K |     126 (0) | 00:00:01 |
| 1  | TABLE ACCESS BY INDEX ROWID BATCHED | ZIGGY        | 20000 |  546K |     126 (0) | 00:00:01 |
|* 2 | INDEX RANGE SCAN                    | ZIGGY_CODE_I | 20000 |       |      33 (0) | 00:00:01 |
----------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

2 - access("CODE"=42)

Statistics
----------------------------------------------------------
     0 recursive calls
     0 db block gets
   136 consistent gets
     0 physical reads
     0 redo size
670725 bytes sent via SQL*Net to client
   533 bytes received via SQL*Net from client
     5 SQL*Net roundtrips to/from client
     0 sorts (memory)
     0 sorts (disk)
 20000 rows processed

We notice the index is now used and has resulted in a significant reduction of consistent gets (just 136, down from 9,354). The query is now much much more efficient than it was previously.

An issue here though is that we have to re-org the entire table in order to address the clustering of any new rows that are subsequently added with standard DML (as the Clustering Attribute is ineffective in this case).

Wouldn’t it be nice if we could easily convert this table (and associated indexes) to be partitioned and hence only have to concern ourselves with subsequently reorganising just those partitions that have new/changed data.

Topic of my next post 🙂

12.2 Index Advanced Compression “High” Part IV (The Width of a Circle) March 10, 2017

Posted by Richard Foote in 12c Rel 2, Advanced Index Compression, Index Compression, Oracle Indexes.
1 comment so far

DAVID_BOWIE_THE+MAN+WHO+SOLD+THE+WORLD-291998

A quick post (for me) with a long weekend beckoning…

In Part I, Part II and Part III of looking at the new Index Advanced Compression level of “High”, we discussed how it can significantly decrease the size of your indexes in a manner not previously possible. This can result in significant reductions of index storage and the potential subsequent reduction in IO and memory related overheads.

This is of course good.

However, if you have applications which have tables/indexes that are very sensitive regarding DML performance, you need to exercise caution before compressing indexes in this manner. This is due to the extra CPU overheads and file IO wait times that can result in maintaining the highly compressed index structures.

To quickly illustrate this point, let’s first look at the timings when performing DML with an associated index that has no compression:

SQL> create table bowie (id number, code number, name varchar2(42));

Table created.

SQL> create index bowie_code_idx on bowie(code) nocompress;

Index created.

SQL> insert into bowie select rownum, rownum, 'ZIGGY STARDUST'
from dual connect by level <= 1000000;

1000000 rows created.
Elapsed: 00:00:06.61

SQL> commit;

Commit complete.

SQL> update bowie set code = 42 where id between 250000 and 499999;

250000 rows updated.
Elapsed: 00:00:12.91

If we now repeat the same demo, but this time with an index that’s compressed with advanced compression set to “HIGH”:


SQL> drop table bowie;

Table dropped.

SQL> create table bowie (id number, code number, name varchar2(42));

Table created.

SQL> create index bowie_code_idx on bowie(code) compress advanced high;

Index created.

SQL> insert into bowie select rownum, rownum, 'ZIGGY STARDUST' from dual connect by level <= 1000000;

1000000 rows created.
Elapsed: 00:00:39.08

SQL> commit;

Commit complete.

SQL> update bowie set code = 42 where id between 250000 and 499999;

250000 rows updated.
Elapsed: 00:01:09.83

We see there’s a significant increase in timings when both inserting into the table and when updating the highly compressed indexed column.

Therefore you need to consider the detrimental impact on DML performance due to the additional resources required in maintaining the highly compressed indexes, as it might offset the potential benefits of having the smaller index structures. Your mileage may vary.

More to come 🙂