Apache AVRO - Data Serialization Framework

AVRO is an Apache open source project for data serialization and data exchange services for Hadoop .Using Avro (which functions similar to systems such as Apache Thrift, Protocol Buffers- Google's) data can be exchanged between programs written in any language. Avro is gaining new users compared to other popular serialization frameworks, for the reason that many Hadoop based tools support Avro for serialization and De-serialization.

Before we get into the features lets understand about Serialization & De-serialization.

Serialization means turning the structured objects into a bytes stream for transmission over the network or for writing to persistent storage.

De-serialization is the opposite of serialization, where we read the bytes stream or stored persistent storage and turns them into structured objects.

The serialized data which is in a binary format is accompanied with schemas allowing any application to de serialize the data.

Some Features Of Avro

• Avro serialized data doesn't require proxy objects or code generation (unless desired for statically-typed languages). Avro uses definitions at runtime during data exchange. It always stores data structure definitions with the data making it easier to process rather than going for code generation.

Application Roles in OBIEE

Default OBIEE Application Roles
Application Role	LDAP Group	Description
BIConsumer	BIConsumers	Base-level role that grants the user access to OBIEE analyses, dashboards and agents.
BIAuthor	BIAuthors	All BIConsumer rights, grants and permissions + allows users to create new analyses, dashboards and other BI objects
BIAdministrator	BIAdministrators	All BIAuthor and BIConsumer rights, grants and permissions + allows the user to administer all parts of the system.

By default, OBIEE 11g have three application roles:

• BIConsumer : The base-level role that grants the user access to existing analyses, dashboards and agents, allows them to run or schedule existing BI Publisher reports, but not create any new ones. That means user can view the existing reports and dashboards.
• BIAuthor : A role that allows users to create new analyses, dashboards and other BI objects. This role will recursively granted the privileges of BIConsumer role.
• BIAdministrator : This role has the privileges of BIAuthor &BIConsumer roles. In addition to that this role allows the user to administer all parts of the system, including modifying catalog permissions and privileges. This user is the most privileged user in OBIEE environment.

In some cases, a need may arise to add another role that suits well between the BIConsumer and BIAuthor roles; one called BIAnalyst, that allows users to create and edit analyses, but not creates new dashboards.

Informatica Scenario: How to Load Data in Cyclic Order using Informatica

Informatica Scenario:

To Load source records from DEP table to target in a cyclic order.

In this scenario  we will load the records present in the DEP table to Flat files in a cyclic order such that the first record will go to TargetFile01 ,2^nd to TargetFile 02 ,3^rd to Target File03 and 4^th record again to TargetFile01.

For implementing this we can make use of sequence generator  with properties as given below.

You can route the records based on the NEXT VAL of sequence Generator using a Router as given below.

The overall mapping will look like this.

Below is the expected format in which target files will be created.

Please read more of such scenarios at http://dwhlaureate.blogspot.in/2013/02/informatica-scenarios-based-questions.html

How To Add Auto Increment In Oracle- IDENTITY column in Oracle 12c

Lets create employee table for this purpose.

SQL> CREATE TABLE EMP01

(

EMPID NUMBER,

NAME VARCHAR2(50),

DEPT VARCHAR2(20)

);

Next steps is to create oracle sequence to generated the id values.

SQL> CREATE SEQUENCE EMPID_SEQUENCE

START WITH 1

INCREMENT BY 1;

Next Step is to create Trigger to assign the values from sequence to EMPID column.

CREATE OR REPLACE TRIGGER EMPID_TRIGGER

BEFORE INSERT ON EMP01

FOR EACH ROW

BEGIN

SELECT EMPID_SEQUENCE.nextval INTO :NEW. EMPID  FROM dual;

END;

/

Now we will try insertng few values:

SQL> INSERT INTO EMP01 (NAME, DEPT) VALUES ('RON',’ABC’);

1 row created.

SQL> INSERT INTO EMP01 (NAME, DEPT) VALUES ('VICTORIA',’XYZ’);

SQL> SELECT * FROM EMP01;

EMPID NAME DEPT

---------- ------------------------------

1 RON ABC

2 VICTORIA XYZ

You can find that the EMPID getting incremented by 1.
Now there is a new feature available in on Oracle 12c version:IDENTITY column using which we can implement the same auto increment feature.