java - ExecutorService with multiple threads not working normally but works fine in debug mode -


basically have process large csv file 1 million records using multi threading.

i created class ingestioncallerthread

public class ingestioncallerthread {  public static void main(string[] args) {      try {         int count = 0;         inputstream ios = ingestioncallerthread.class.getclassloader().getresourceasstream("aa10.csv");          byte[] buff = new byte[8000];          int bytesread = 0;         bytearrayoutputstream bao = new bytearrayoutputstream();          while ((bytesread = ios.read(buff)) != -1) {             bao.write(buff, 0, bytesread);         }          byte[] data = bao.tobytearray();          bytearrayinputstream bin = new bytearrayinputstream(data);         bufferedreader fileinputstreambufferedreader = new bufferedreader(new inputstreamreader(bin));          while ((fileinputstreambufferedreader.readline()) != null) {             count++;         }         bin.reset();          int numberofthreads = 12;         int rowsforeachthread = count / numberofthreads;         int remrows = count % numberofthreads;         int startposition = 0;         system.out.println(count);         executorservice es = executors.newcachedthreadpool();         (int = 0; < numberofthreads && startposition < count; i++) {             if (remrows > 0 && + 1 >= numberofthreads)                 rowsforeachthread = remrows;              ingestionthread ingthread = new ingestionthread(bin, startposition, rowsforeachthread);             es.execute(ingthread);             startposition = (startposition + rowsforeachthread);         }         es.shutdown();         if (es.isterminated()) {             system.out.println("completed");         }         // t2.start();     } catch (ioexception e1) {         // todo auto-generated catch block         e1.printstacktrace();     }      catch (exception e) {         // todo auto-generated catch block         e.printstacktrace();     }  }  } 

which use call runnable class have implemented

public class ingestionthread implements runnable {  inputstream is; long startposition; long length;  public ingestionthread(inputstream targetstream, long position, long length) {     this.is = targetstream;     this.startposition = position;     this.length = length; }  @override public void run() {     // todo auto-generated method stub     int currentposition = 0;     try {         is.reset();     } catch (ioexception e1) {         // todo auto-generated catch block         e1.printstacktrace();     }     bufferedreader fileinputstreambufferedreader = new bufferedreader(new inputstreamreader(is));     if (startposition != 0) {            string line;         try {             while (((line = fileinputstreambufferedreader.readline())) != null) {                 if (currentposition + 1 == startposition)                     break;                 currentposition++;             }         } catch (ioexception e) {             // todo auto-generated catch block             e.printstacktrace();         }     }     try {         int execlength = 0;         string line;         while ((line = fileinputstreambufferedreader.readline()) != null && execlength < length) {             system.out.println(line);             execlength++;         }     } catch (ioexception e) {         // todo auto-generated catch block         e.printstacktrace();     } }  } 

i tested small csv file of 20 records. issue when debug class records getting printed. when run class 15 records read, 12 records read. not sure issue. appreciated. in advance.

the cause of problems have number of threads reading different bufferedreader objects wrap shared bytearrayinputstream. there no synchronization, , means different threads reading sections of stream other threads supposed reading.

each of threads needs own bytearrayinputstream.


Comments

Popular posts from this blog

php - Permission denied. Laravel linux server -

google bigquery - Delta between query execution time and Java query call to finish -

python - Pandas two dataframes multiplication? -