Thursday, November 12, 2015

Understand Glassfish JDBC configuration by inspecting JDBC DataSource API

When configure DataSource on JEE container Glassfish, There are 2 things need to be set, JDBC Resource and JDBC Connection Pools.

image

What's the difference and why are these two items? What's the relation between them? We can answer that by inspecting the Java JDBC API, mainly in package javax.sql

1. How to connect to DB

To work with DB in Java, the application must have a connections  (java.sql.Connection) to the database.  There are 2 ways to get a connection in JDBC:

  • Method 1, using static method, DriverManager.getConnection(url). This is a very old way, still valid, only used in small standalone applications, doesn't support pooling.
  • Method2, using DataSource interface. Recommended. In most case work with JNDI lookup to get one from a container.

2. The hierarchy of DataSource

 

jdbc

There are some points need to be noticed about this diagram:

They are all interfaces. No concrete class at all.

There are 3 interfaces related to data source, DataSource/ConnectionPoolDataSource/XADataSource.

Application Java code will ONLY come into contact with the green part. DataSource and Connection.

ConnectionPoolDataSource instance can create PooledConnection instance(pooled...you know,name tells all), XADataSrouce can create XAConnection (for distributed transaction)

The ConnectionPoolDataSource, although may sounds like, but is not a Child of DataSource. In other words, A  ConnectionPoolDataSource instance can't be simply casted to DataSource instance. So inside a DataSource instance, the real work will somehow be delegate to a ConnectionPoolDataSource/XADataSouce.

Same thing applied to Connection. The Connection has nothing to do with PooledConnection, although they looks similar. No casting but delegation.

Up to know you should understand the relationship between JDBC Resource and JDBC Connection Pools in Glassfish Administration GUI. Think JDBC Resource as DataSource and JDBC Connection Pools  as ConnectionPoolDataSource.

3. Using MySQL in Glassfish

Let's use MySQL's official developer guide on "Using Connector/J with GlashFish" as a testify. On this document, the configuration has 2 steps:

  • step1. Creating a Connection Pool, filling in  information like DB's IP address, username and password for make "real" connections to database.
  • step2. Create a JDBC Resource. choose a pool created in step 1 and set a JNDI name for it.  We can say the JDBC Resource HAS-A pool.

The steps can pretty much demonstrate the hierarchy of DataSource in JDBC API, which once get understood will make the configuration steps make sense and easy to remember.

Tuesday, November 10, 2015

Break down class "Files" of java 7 NIO.2

Since Java 7, File related IO operation has been redesigned. The following 3 classes/interfaces are key for file related operations: Path,Paths, and Files. They are all in package java.nio.file. Among all these 3, Files is the most fundamental one.

0. Brief about Path and Paths

Paths is a concrete classes which only has method get(...) to create Path instance. so Paths is just a factory class whose only usage is get a Path instance from a input String or URL to static method get(...)

Path is an interface designed to replace old-styled java.io.File class.  First thing to notice is that Path is not a class but a interface, can't create instance with keyword new. So Paths was introduced for doing the initial.  Second thing need to know about Path is its all method will not do all really I/O, just string manipulation. All methods of Path will not touch anything on the HD. For example method to normalize a path "/usr/local/../bin/../tmp" to path "/usr/tmp" located in class Path.

Then who make the "real" I/O operation? The class Files - this beauty has it all! 

1. Anatomy of class Files

Files is a concrete class only has plenty of static methods. All most all methods need Path instance as its first parameter.

java_nio_file

There seems a lot of stuff, but take it easy, let's go through them step by step. The functions can be categorized in 4 groups.  Also The diagram contains most used methods of Files, but not all. The methods' signature are simplified just for easy to understand without bothering too many details.

  • group 1, basic manipulation. Create/Delete/Rename/Move a file or directory on the file system.
  • group 2, go through directory. 2 methods used to list directory contents without and with subdirectories respectively.
  • group 3, read/write content of file.
  • group 4. get/set file related attributes (the rest parts on the above diagram with pink/blue/green colors)

1.1 Basic I/O manipulation

The methods in these group are pretty self-explained.  By utilizing methods in this group you should be able solve followings:

  • How to create/copy/rename/move/delete a single file?
  • How to create/copy/delete empty directory, rename/move non-empty directory?

There is no single method can copy/delete non-empty directory, has to fulfill that with methods in group 2.

1.2 Go through directory

There are 2 methods in this group. Method newDirectoryStream() works just like command-line 'ls' or 'dir', not recursively deep into subdirectories. On the contrary, walkFileTree() recursively go though every subdirectories.

How to list a directory, only one tier?

  public static void listDirectory(String dir) throws IOException {
Path path = Paths.get(dir);
DirectoryStream<Path> dirStream = Files.newDirectoryStream(path, "*.java");

for (Path p:dirStream) {
// do something will every file/subdir in this directory
System.out.println(p.getFileName());
}
}

How to go through a direcoty recursively, include all subdirectories ?

  public static void goThoughDirectoryRecursively(String dir) throws IOException {
Path path = Paths.get(dir);
Files.walkFileTree(path, new SimpleFileVisitor<Path> () {
@Override
public FileVisitResult visitFile(Path path, BasicFileAttributes attrs) throws IOException {
System.out.println(path); // print out name of every file/directory
return FileVisitResult.CONTINUE;
}
});
}

The walkFileTree() usage is a standard incarnation of "Visitor" design pattern.


1.3 Read/write file contents


These group are for reading or writing contents of file. Use Reader/Writer for text file and InputStream/OutputStream for binary file. Also there are helper methods to read all contents at once for small files.

  public static void readSingleFile(String fileName) throws IOException {
Path path = Paths.get(fileName);
BufferedReader reader = Files.newBufferedReader(path, StandardCharsets.UTF_8);

// print the file content to screen
String line;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
reader.close();
}

1.4 Get/set file attributes


The file and directory on file system can have many attributes, some basic attributes like creation/access/modification time. some of the attributes are specific to windows, like is the file hidden? Some of them are specific to Unix, like is the file writable to other users?


There are 3 ways to get/set file attributes:

  • Use simple shortcut static methods provided by class Files to test and set  (pink on the diagram)
  • Use attributes related classes  like XXXFileAttributes and XXXFileAttributeView (blue on the diagram, see explanation below)
  • Use powerful shortcut static methods getAttribute()/setAttribute() (green on the diagram, see blow)

1.4.1 simple shortcut


The methods for simple shortcuts are ....simple. yes, they are strait forward.  But simple shortcuts don't cover all attributes just some most used ones. Also you may already notice that fewer methods for set than get. ( There is a 'hole' on up-right corner). Shortcut methods in fact just use other attributes related classes we will see below for simplicity.


1.4.2 attributes related classes


Check the blue part on the diagram, left 3 named like XXXFileAttributes, which are read-only. Right 3 named like XXXFileAttributeView are used for file attribute modification.  The XXX here in class name is called view-name, we'll see the usage of view-name very soon.


Here is code snippet to read file attributes on windows.

  // test if a file is read-only on Windows
public static boolean isFileReadOnly(String fileName) throws IOException {
Path path = Paths.get(fileName);
DosFileAttributes dosAttr = Files.readAttributes(path, DosFileAttributes.class);
return dosAttr.isReadOnly();
}

Here is code snippet for modify file attributes on Linux

  // set file permission to "rw-r--r--" on Linux
public static void modifyFilePermission(String fileName) throws IOException {
Path path = Paths.get(fileName);
PosixFileAttributeView posixAttrView = Files.getFileAttributeView(path, PosixFileAttributeView.class);
posixAttrView.setPermissions(PosixFilePermissions.fromString("rw-r--r--"));
}

1.4.3 powerful shortcut


Unlike simple shortcuts, powerful shortcuts only have a pair of getAttribute(path,attributeString...)/setAttribute(path,attributeString...) , but can cover all file attributes. The format of string parameter attributeString is  [view-name:]attribute-name

  • The default value for view-name is basic
  • The attribute-name is in lower case.
  • Need type-casting because get/set only return or use Object instance as values 

We have a code snippet above isFileReadOnly(...) to test if a file is read-only on windows. Here is the equivalent.

  // test if a file is read-only on Windows 
public static boolean isFileReadOnly(String fileName) throws IOException {
Path path = Paths.get(fileName);
return (boolean) Files.getAttribute(path, "dos:readonly");
}

2.Recap


Get a clear view of class Files will give you a good understanding of file related I/O functions in Java 7 + NIO.2.  Again let's review these 4 groups :

  • group 1, basic manipulation. Create/Delete/Rename/Move a file or directory on the file system.
  • group 2, go through directory. 2 methods used to list directory contents without and with subdirectories respectively.
  • group 3, read/write content of file.
  • group 4. get/set file related attributes (the rest parts on the above diagram with pink/blue/green colors)

How to understand < ? super Child> in Java Generics

In Java, polymorphism does NOT apply to generic types. So suppose class Child extends from class Parent, then the following 2 lines, only the first can pass compilation.

  Parent var = new Child();  // compiler is happy with this
ArrayList<Parent> myList = new ArrayList<Child>(); // compilation error

The way to bring the idea of inheritance into Java generic types is using <? extends Parent> or <? super Child>. In this article we'll see how to understand the meaning of syntax <? super Child> in Java generics. 


Check another article on how to understand syntax <? extends Parent> in Java generics.


0. Clarify concept

  interface Animal{ }
class Dog implements Animal { }
class Cat implements Animal { }

public void test() {
// make sure you understand these 2 lines
List<Animal> pList = new ArrayList<Animal>();
pList.add(new Dog()); // It's fine
}

There is a List of type Animal variable pList. Can a Dog instance be added to this list? The answere is yes, because the Dog IS-A Animal;


1. Basic meaning


<? super Child> refer to any class that see class Child as its descendent.


<? super Child> can be used for variables definition and  method's parameter, but most used in latter.


2. Make a collection (kind of) write-only


If generic types' syntax <? super Child> used as method's parameter definition, then the parameter is (kind of) write-only. Usually it's used with a collection parameter.


Why ? Let's image you have a method  addDogToList defined like

  void addDogToList (List<? super Dog> myList) {
myList.add(new Dog()); // design to add Dog to collection

Cat obj = (Cat) myList.get(0); // Compile OK, but risky!
// Never explicitly cast type
// when using generic types
}

The addDogToList has a paramenter myList as type List<? super Dog>.  So the following usage are both correct since Dog instance can be add to a either Dog list or Animal List (see charter 0. clarify concept above)

  addDogToList(new ArrayList<Dog>());
addDogToList (new ArrayList<Animal>());

we just explained why "write" to that list is OK, but why "write-only"? Technically speaking, you can read elements from collection marked by , but the return type is Object, which means you can cast it to any Class you want and the compiler will let you pass anyway, like what we did above, we cast element to type Cat, but during runtime that normally means a disaster. Furthermore, the whole meaning of Generics is to eliminate type casting for the sake of type-safe. That's why call it "kind of write-only" (you can read, but don't)


3. Recap


When use a collection variable with <? super Child> style, it means " Hey, I'm going to add Child instance into this collection, just make sure the argument passed in can hold new Child instance."

Monday, November 9, 2015

How to understand < ? extends Parent> in Java Generics

In Java, polymorphism does NOT apply to generic types. So suppose class Child extends from class Parent, then the following 2 lines, only the first can pass compilation.

  Parent var = new Child();  // compiler is happy with this
ArrayList<Parent> myList = new ArrayList<Child>(); // compilation error

The way to bring the idea of inheritance into Java generic types is using <? extends Parent> or <? super Child>. In this article we'll see how to understand the meaning of syntax <? extends Parent> in Java generics. 


Check another article on how to understand syntax <? super Child> in Java generics.


1. Basic meaning


<? extends Parent> means any class that see class Parent as its ancestor. Parent can be either a class or an interface. (yes, interface is OK although keyword extends is used)


<? extends Parent> can be used for variables definition and  method's parameter


2. Make a collection read-only


If generic types' syntax <? extends Parent> used as method's parameter definition, then the parameter is read-only. Usually it's used with a collection parameter, and that collection is read-only inside the method.


Why ? Let's image you have a method printName defined like

  interface Animal{
String getName();
}
class Dog implements Animal { // omit getName implementation }
class Cat implements Animal { // omit getName implementation }

void printName(List<? extends Animal> animals) {
// Good to read from List
for (Animal animal : animals) {
System.out.println (animal.getName());
}

// Error when write to list, compile fail
animals.add(new Dog());
}

The printName has a paramenter animals as type List<? extends Animal>.  So the following usage are both correct since Dog and Cat are all subclass of Animal.

  printName (new ArrayList<Dog>());
printName (new ArrayList<Cat>());

Now let answer the question about why read-only. In method printName, every element out of list can be guaranteed IS-A type Animal, that's why read from list is OK. But when try to add new element,  the compiler has no idea the input list animals has Dog or Cat in it. Because anyone can call this printName method with eight Dog list or Cat list. This information is unknown to compiler, so all the java compiler can do is  fail there to avoid making severe mistakes adding  a Dog instance into a Cat list.


3. Recap


When use a collection variable with <? extends Parent> style, it means " Hey, I'm gonna use elements in this collection and make sure every one in this collection fulfill IS-A type Parent requirement. But I promise will never ever try to add anything back into this collection"

Thursday, November 5, 2015

Glob in Java file related match

Globs are not regular expression. Glob is simpler and earlier than regular expression. For historical reasons, glob are more used as file name or file path filtering.

In Java 7 NIO package has 2 places that glob appears as file names or file path filter. One is when you create a PathMatcher to test if a java.nio.file.Path instance matches a patten like

Path path = Paths.get("abc.java");
PathMatcher matcher = FileSystems.getDefault().getPathMatcher("glob:*.java");
boolean isJavaFile = matcher.matches(path); // true

The other is when create a DirectoryStream to iterate all files and subdirectories( not including files under subdirectories, just like the command 'dir' or 'ls')

Path dir = Paths.get("/tmp");
try (DirectoryStream<path> stream = Files.newDirectoryStream(dir,"[vt]*")) {
for (Path path : stream) {
System.out.println(path.getFileName()); // only files start with 'v' or 't'
}
}

Here are the rules for glob:


  • * match any char except a directory boundary
  • ** match any char include a directory boundary
  • ? match any ONE char
  • [] same as regular express, like [0-9] match any ONE digit.
  • {} match a collection of patten separated by comma ','. Such as {A*, b} means either a string start with 'A' or a single char 'b'.

More examples:

pathglobmath result
/tmp/src/main/Demo.java*.javafalse
/tmp/src/main/Demo.java**.javatrue
/tmp/src/main/Demo.java/tmp/*/*.javafalse
/tmp/src/main/Demo.java/tmp/**/*.javatrue
/tmp/src/main/Demo.java/tmp/**/[dD]*.javatrue
/tmp/src/main/Demo.java/tmp/**/[aA]*.javafalse

Tuesday, November 3, 2015

A brief in Java string format

When to format a Java String using System.out.printf or System.out.format, The syntax for the format is :

%[arg_index$][flags][width][.precision]<conversion_char>

Here are more explanation:
  • arg_index    An integer with suffix '$'. begin with 1, not 0. The position of the args.
  • flags 
    - Left-align, default is right-aligned
    + Include +/-before number
    0 Pad with zeros before number
    , User comma to separate number
    ( Enclose negative numbers in parentheses. '-' will not display
  • width  The width of the printed String, very useful to get table like output
  • precision  For float/double, specify the digits after "."
  • Conversion_char
    b boolean
    c char
    d integer
    f float/double
    s string

Below is a simple example for demostration

    System.out.printf("<%-3s><%6s><%10s><%10s>\n","Id","Gender","Name","Balance");
System.out.printf("<%3$03d><%2$6s><%1$10s><%4$+10.2f>\n", "Tom","M",1,688.972); // arg_indx
System.out.printf("<%03d><%6s><%10s><%+10.2f>\n", 2,"M","Jerry",-12.345);
System.out.printf("<%03d><%6s><%10s><%(10.2f>\n", 3,"F","Rose",-12.345);

The "<"  and ">" are added to make the border visible. The outputs are:

<Id ><Gender><      Name><   Balance>
<001>< M>< Tom>< +688.97>
<002>< M>< Jerry>< -12.35>
<003>< F>< Rose>< (12.35)>

Friday, October 30, 2015

How to use embedded Java DB (Derby) in maven project

For now, Java DB is actually just Apache Derby with a different name.  In the following of the article, we will call it Derby. It  comes with JDK installation. ( Although what's normally used in maven project is not the same binary install in local JDK directory)

Using embedded Java DB means the database will run in the same JVM as your application. The Java DB engine actually gets started when you try to connect to it by JDBC. When the application exits, the database also exits. If you choose to run the Java DB total in memory, when the JVM stops, the data will be gone. Or you can choose to store the data on local file system to make them usable during multiple runs.

Java DB (Derby) is mostly used for convenience in development. No external database is needed even you have code need to play with RMDB.

0. What you need

  • JDK 6+ (JDK 7 in this demo)
  • Maven 3.2 +

1. POM file

There's only one dependency needed to use Derby database.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>com.shengwang.demo</groupId>
<artifactId>javadb-derby-embedded-basic</artifactId>
<version>1.0</version>
<packaging>jar</packaging>

<name>javadb-derby-embedded-basic</name>
<url>http://maven.apache.org</url>

<dependencies>
<dependency>
<groupId>org.apache.derby</groupId>
<artifactId>derby</artifactId>
<version>10.8.3.0</version>
</dependency>
</dependencies>

<!-- Use Java 1.7 -->
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.5.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
</plugins>
</build>
</project>

Since the version of Java DB comes with JDK 7 is 10.8.3.x, we also use 10.8.3.0 in our maven project. If just using in-memory database(see below), the version used in maven actually doesn't matter. But if database file stores on file system and you want to check the database content after application finishes, you'd better use the same version as from JDK, so the 'ij' tool in $JAVA_HOME/db/bin can open the database without version conflicts.


2. When the embedded database starts


The database starts when you java code try to connect to it by using standard JDBC. How the derby work depends on the way to connect to it, or in other words, depends on the connection url. Suppose we need to connect to a database named 'demo'.


In-memory database, url looks like: jdbc:derby:memory:demo;create=true


'demo' is the database name and can be any string you choose, "memory" is a key word to tell Derby  to goes to all-in-memory mode.


File-based database, url looks like: jdbc:derby:c:\Users\shengw\MyDB\demo;create=true


'c:\Users\shengw\MyDB\demo' is the directory to save database files on local file system. (On windows its actually jdbc:derby:c:\\Users\\shengw\\MyDB\\demo;create=true because of the String escaping)


'create=true' is a Derby connection attribute to create the database if it doesn't exist. If use in-memory database, this attribute is mandatory.


3. A complete example


This is a complete hello world level example using embedded Derby database in Maven project. The HelloJavaDb.java lists below.

package com.shengwang.demo;

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;

public class HelloJavaDb {
Connection conn;

public static void main(String[] args) throws SQLException {
HelloJavaDb app = new HelloJavaDb();

app.connectionToDerby();
app.normalDbUsage();
}

public void connectionToDerby() throws SQLException {
// -------------------------------------------
// URL format is
// jdbc:derby:<local directory to save data>
// -------------------------------------------
String dbUrl = "jdbc:derby:c:\\Users\\shengw\\MyDB\\demo;create=true";
conn = DriverManager.getConnection(dbUrl);
}

public void normalDbUsage() throws SQLException {
Statement stmt = conn.createStatement();

// drop table
// stmt.executeUpdate("Drop Table users");

// create table
stmt.executeUpdate("Create table users (id int primary key, name varchar(30))");

// insert 2 rows
stmt.executeUpdate("insert into users values (1,'tom')");
stmt.executeUpdate("insert into users values (2,'peter')");

// query
ResultSet rs = stmt.executeQuery("SELECT * FROM users");

// print out query result
while (rs.next()) {
System.out.printf("%d\t%s\n", rs.getInt("id"), rs.getString("name"));
}
}
}

The demo uses derby database, creates a table 'users', inserts 2 rows into the table and prints the query result set.  The whole maven project hierarchy is :


image



After running the HelloJavaDb example, you can verify the database, because we are not using derby in all-in-memory mode.  After running, the database files will appear on local file system like this. 


image


If you connect to the database in command line, you can see the 2 rows add from your Java code. 'ij' is a tool provided by Derby works as a sql client.


image

Powered by Blogger.

About The Author

My Photo
Has been a senior software developer, project manager for 10+ years. Dedicate himself to Alcatel-Lucent and China Telecom for delivering software solutions.

Pages

Unordered List