In an earlier post of mine I had given an example of how to use VTD Xml.We could run into requirements where in we would like to use VTD xml in a multi threaded application.This post provides a working example for integrating VTD xml in a multi threaded application.
The requirement: I have large number of XPATHs that need to be resolved.Due to large number of XPATHS,our application may slow down.So the better way would be to resolve XPATHs in multiple threads.
In the below code an important point that needs to be noted is that after parsing the xml using VTDGen,we store the navigator in a variable by invoking GetNav().The GetNav() call cleans internal state , so we should keep a copy of navigator and pass the “duplicate” of navigator object to each task.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 |
import java.io.File; import java.util.*; import java.util.concurrent.Callable; import java.util.concurrent.ExecutionException; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.Future; import com.ximpleware.AutoPilot; import com.ximpleware.EOFException; import com.ximpleware.EncodingException; import com.ximpleware.EntityException; import com.ximpleware.ParseException; import com.ximpleware.VTDGen; import com.ximpleware.VTDNav; public class MultiThread { public static void main(String args[]) throws InterruptedException, ExecutionException, EncodingException, EOFException, EntityException, ParseException { //str contains the sample xml, one can read an xml from a file as well String str="<library><booked>book</booked> <book id=\"1\"> <title>Googled By God</title> </book> </library>"; //Create 5 threads ExecutorService executor = Executors.newFixedThreadPool(5); List<Task> extractorTasks = new ArrayList<Task>(); //Create VTDGen object and parse the xml VTDGen vg = new VTDGen(); vg.setDoc(str.getBytes()); vg.parse(false); //The GetNav() call cleans internal state , so keep a copy of VTDNav VTDNav vn = vg.getNav(); //Create 100 tasks and queue them for(int i=0;i<100;i++) { //pass the duplicates of navigator. //To all my threads I am passing the same xpath, but one might read the list //of xpaths from a file Task d = new Task(str.getBytes(),vn.duplicateNav(),"/library/book/title"); extractorTasks.add(d); } List<Future<String>> output = executor.invokeAll(extractorTasks); executor.shutdown(); } } class Task implements Callable<String> { String xpath = ""; //ba is the byte array byte [] ba=null; VTDNav vn = null; AutoPilot ap = null; Task(byte[] _ba,VTDNav _vn,String _xpath) { ba = _ba; vn = _vn; xpath = _xpath; } public String call() throws Exception { String title = ""; try { ap = new AutoPilot(vn); ap.selectXPath(xpath); //Get all the titles and print each of those while(ap.evalXPath() != -1) { //getText will return the index of the VTDRecord int titleIndex = vn.getText(); //Get the text of the VTDRecord title = vn.toNormalizedString(titleIndex); System.out.println("Title is "+title); } } catch (Exception e) { e.printStackTrace(); System.out.println("Exception Message is "+e.getMessage()); } return title; } } |