New Startup Firm - DeduXion

I'm proud to announce that today I launched the new website for DeduXion.

We will be posting new projects and developments along the way, so stay tuned in ;)

Q

@DeduXion is almost founded!

Hey everyone

I'm happy to announce that our start-up firm "DeduXion" is almost founded.
DeduXion will be a company that will make software for small and medium-sized construction companies to support their paperwork and inventory systems. We already created some modules in Java for our internship at a construction company, but realized that there could be a bright future in front of us if we actually sold the products.

We just have to fill in some papers and contracts and we're good to go. I'll be keeping you posted on projects and findings. We're going to use some agile developing techniques that seem to be quite effective for smaller IT-companies.

Hope to see you soon!

@Deduxion

[Java] Start Amount of Threads depending on Available Cores

Parallel threading depending on Amount of Cores

Here's a small snippet to keep in mind when you're programming and you've got to do some parallel computing.

static void test() {
int n = Runtime.getRuntime().availableProcessors();
System.out.println(n);
for (int i=0; i<n; i++) {
Thread(new Runner()).start();
}
}

It's not recommended to start your threads like this (I'd suggest an ExecutorService for it), but it's something to remember ;)

Qkyrie

[Java] Execute a terminal-command in Java

Here's an easy way to implement command calls in Java. The program prints the response from the CLI.

import java.io.*;

1. public class TestExec {
2. public static void main(String[] args) {
3. try {
4. Process p = Runtime.getRuntime().exec("cmd /C dir");
5. BufferedReader in = new BufferedReader(
6. new InputStreamReader(p.getInputStream()));
7. String line = null;
8. while ((line = in.readLine()) != null) {
9. System.out.println(line);
10. }
11. } catch (IOException e) {
12. e.printStackTrace();
13. }
14. }
15. }

Qkyrie

Web Designers vs Web Developers - Infograph

Totally had a big laugh when I saw this one. Especially for us, coders, this is really, like, striking.

Web Designers vs Web Developers is brought to you by Wix.com
Use creative design to make a Free Website
You are most welcome to share this infographic with your audience.

[Java] Format a Float @ 2 decimals precision

Here's a small code example of how you could format a float to be displayed at 2 decimals precision (only when necessary).

1. import java.text.NumberFormat;
2. ...
3. NumberFormat nf = NumberFormat.getInstance();
5. nf.setMaximumFractionDigits(2);
6. nf.setMinimumFractionDigits(0)

To actually format a float, one could simply do:

1. nf.format(myFloat);

It's as simple as that ;)

@Qkyrie

[Java] Resizing a BufferedImage

Here's a small piece of code that might aid you in resizing a BufferedImage. Most methods I saw on the internet just took the BufferedImage and resized it to fit the screen. This method will return a new bufferedImage with the new width and height. 

1. public static BufferedImage resize(BufferedImage img, int newW, int newH) {  
2.         int w = img.getWidth();  
3.         int h = img.getHeight();  
4.         BufferedImage dimg = dimg = new BufferedImage(newW, newH, img.getType());  
5.         Graphics2D g = dimg.createGraphics();  
6.         g.setRenderingHint(RenderingHints.KEY_INTERPOLATION,     RenderingHints.VALUE_INTERPOLATION_BILINEAR);  
7.         g.drawImage(img, 0, 0, newW, newH, 0, 0, w, h, null);  
8.         g.dispose();  
9.         return dimg;  
10.     }  

New Project - EmailCrawler in JavaFX With JScrape.

This week I decided to port my gui from my emailcrawler I had written before
(In pure Java)  to JavaFX.

Unfortunately, I wasn't satisfied with the code I had written over a year ago.
I easily discovered that, a year ago, I wasn't able to write a single class without
making obvious epic failures.

That's why I decided to use my JScrape framework to create an entirely new project.

Release 1 is soon to come. I reckon by friday.

I'll keep in touch

Qkyrie

Performance Showdown - Map<K,V> vs. List<T> Java

Time for another showdown. This time, it's the Java Map Interface vs the List Interface. In what cases are Maps faster than Lists?

Accessing complexity


Actually, if you don't want to entirely read this article, you should know that in almost all cases accessing a list is faster.

When you're accessing a hashmap, you'll first create a hash representation of your key, get to the right "bucket" and then get the value from an implemented list, whereas accessing a value from an ArrayList will merely fetch the value from the list.

After this fact, you might not want to use Maps, but don't worry, there's more..

Two Entirely Different Datastructures


You should never forget that both Map and List are two entirely different datastructures, each with its own advantages.

For example, the keys for a Map aren't linear. Anything can be the key, which becomes very handy when dealing with Strings as keys.

Remember this:

If you need a List, use a list. If you need a Map, use a Map. If at any time you can pick one of both data structures - for example when dealing with consecutive Integers as keys - you're probably better off picking the ArrayList implementation.

But there's more

Remember ArrayList isn't the only List implementation! Choosing the right interface is a first step to choosing the right datastructure, but sometimes a lot of performance can be gained by using another implementation of the chosen interface.

If you frequently add elements to the beginning of a List or iterate over the List to delete elements from its interior, you should consider using LinkedList.

Lists: ArrayLists, LinkedList, Vector (synchronised list)
Maps: HashMap, TreeMap, LinkedHashMap

Finally, there's also a Set interface, used when you don't need access by key, and want to use methods like "contains()"

Sets: HashSet, TreeSet, LinkedHashSet


Comments
If you'd like to add something to this, or you want to correct me on something, feel free to leave a comment below or on my twitterpage!


@Qkyrie and David

Desktop Application Vs. Web Applications

Ok, here's the deal. I love web applications, there's no doubt about that. I like working with Java EE, even Rich Internet Applications like silverlight apps belong to technologies I adore, but due to recent events, I'd like to point out why desktop applications aren't likely to entirely disappear.


Security


When dealing with information that should never be leaked anywhere, BI for example, you should really watch out creating web applications.

Web applications are all fun and games until information gets leaked. If you create a web application, security should be one of your main concerns

Speed


Sometimes you're dealing with a large amount of data. You'll have to keep the difference in mind between traffic and server load.
If you create a pure webbapplication as a thin client, you'll create a massive server load (of course, only when you use a lot of information). On the other hand, if you create a thick client, you'll be having a lot of traffic. Try using a system that inherits both features.


Connectivity


What if the website is down?
Ok, you might say the same thing when using a normal client-server + backend database model. Though, one must not forget, if you host a webapplication, 9 out of 10 times it is hosted on an external server, by a third party. Client/Server models will run on mostly internal clients and servers. Therefore, you have more control over your projects.

Cost


Everything is related to money, creating money and avoiding spending it. First of all, hosting a webapplication can become pricy for what it actually is. Some older server can't handle an advance web application server (not talking php, more .net here). Sometimes it's just cheaper to build a small client-server java model, which uses a minimum amount of server resources. You'll have to check what the advantages are against buying new hardware or hosting it somewhere else.

Leave comments or additional reasons if you have any. I'll make revisions if you keep posting opinions ;).

~ Qkyrie

[Java] changing font size of JLabel

Change the fontsize of a jLabel.

Here's a very short example on how to change the size in a jLabel.

Of course, to do this, you'll have to include the Font class.

Code:

import java.awt.Font;

Here's the short code to change only the size!

Code:

Font curFont = myLabel.getFont();
myLabel.setFont(new Font(curFont.getFontName, curFont.getStyle, 18));

@Qkyrie

[Java] Set System Look and Feel

What is the Java "Look And Feel" 

The architecture of Swing is designed so that you may change the "look and feel" (L&F) of your application's GUI (see A Swing Architecture Overview). "Look" refers to the appearance of GUI widgets (more formally, JComponents) and "feel" refers to the way the widgets behave.

Set the system Look and Feel

public static void main(String[] args) {
try {
// Set System L&F
UIManager.setLookAndFeel(
UIManager.getSystemLookAndFeelClassName());
}
catch (UnsupportedLookAndFeelException e) {
// handle exception
}
catch (ClassNotFoundException e) {
// handle exception
}
catch (InstantiationException e) {
// handle exception
}
catch (IllegalAccessException e) {
// handle exception
}


//program
}

@Qkyrie

Online #ProxyScraper, based on the #JScrape Framework.

Free ProxyScraper

So here's a little project I created in a small timespan. It was simply because someone wanted an automated proxyscraper. Simply to be able to prove it to that person, I created this PoC RIA.

This project will be updated, not sure when, but I keep updating my projects from time to time. You can follow @Qkyrie to see how my projects evolve.

This application was written with a Java FX frontend and a Java backend using the jscrape framework .

It is possible I'll be posting some small bits of code to show you how to fix some Java FX things, but I'm not really a master at creating RIA's, so it might take some time before I have the time.

You'll have to accept the Java certificate to make it trusted, but don't fear, it's not a Java drive by. You can download the source code ;)

@Qkyrie

[Java] Extracting information from websites - IMDB example

Disclaimer: this tutorial was not written by me, Qkyrie. In fact, it was written by Deque, a high-quality member on hackforums.net.


What am I about to learn from this tutorial?

The first part covers the search after movietitles in IMDB. Because websites are heterogeneous and can't be treated with the same code, I particularly give attention how you will find solutions for similar problems. With this you should be able to write your own filterprograms.

Which requirements are there to understand this tutorial?

I assume that you are an light advanced Java programmer. You should have previous knowledge with stringhandling and streams. Advanced OOP knowledge is not necessary.

I recommend to work through this tutorial for string manipulation, if you have problems with it: http://www.esus.com/docs/GetIndexPage.jsp?uid=224

For network streams look at this site: http://oreilly.com/catalog/javaio/chapter/ch05.html

How do I use a search engine with Java?

I suggest you first view the site whose data you want to get. How does the URL look like after starting a search? What are the parameters given through the URL? In case of IMDB a searchstring may look like this: http://www.imdb.com/find?s=all&q=die+hard

After s= follows a searchoption. s=all searches for all, that the database can give you, e.g. actors, characters. If you want to limit your search to movietitles, you have to set s=tt(you figure this out by trying). q stands probably for query. That's where you have to put your searchstring in which single words are divided by +.

Searching with google after a solution, you may find something like that:
http://www.iks.hs-merseburg.de/~uschroet...8b320dcd1e

Code:

package com.tutego.insel.net;

import java.net.URL;
import java.net.URLEncoder;
import java.util.Scanner;

public class YahooSeeke

{
public static void main( String... args ) throws Exception
  {
    String search = "the green mile";

    if ( args.length > 0 )
    {
  search = args[ 0 ];

  for ( int i = 1; i < args.length; i++ )
    search += " " + args[ i ];
    }

   search = "p=" + URLEncoder.encode( search.trim(), "UTF-8" );
   URL u = new URL( "http://de.search.yahoo.com/search?" + search );

   String r = new Scanner( u.openStream() ).useDelimiter( "\\Z" ).next();
   System.out.println( r );
}
}

This works fine with the given example. But if you try to use http://www.imdb.com/find? instead of the yahoo-address you will be blocked. You get this message:
Server returned HTTP response code: 403

Looking up the meaning of the response code (e.g. google "HTTP response code"), we get this answer: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

Quote:10.4.4 403 Forbidden
The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated. If the request
method was not HEAD and the server wishes to make public why the request has not been fulfilled, it SHOULD describe the reason for the refusal in
the entity. If the server does not wish to make this information available to the client, the status code 404 (Not Found) can be used instead.

Since we are not allowed to get access, we search for an IMDB API. Not every search engine provides an API, because most of them don't want to allow automatic searches. There is also no API for IMDB, but we are allowed to download text data files: http://www.imdb.com/interfaces
It is not easy handling these text files and it is an disproportional effort for a little program. Therefore we find APIs from other people to make this work easier, e.g.:http://www.deanclatworthy.com/imdb/
With the aid of this webservice you are able to do a search through changing two lines of code: 

Code:

search = "q=" + URLEncoder.encode(search.trim(), "UTF-8");
        URL u = new URL("http://www.deanclatworthy.com/imdb/?" + search);

The output will look like this:

Quote:{"title":"The Green Mile","imdburl":"http:\/\/www.imdb.com\/title\/tt0120689\/","country":"USA","languages":"English&nbsp;,French","genres":"Crime,Drama,Fantasy,
Mystery","rating":"8.4","votes":"211313","usascreens":2875,"ukscreens":340,"year":"1999",
"stv":0,"series":0}

We can gather a lot of information with some string processing. But we get not more than text and we have to trust that this service will be continued. It would be much better to get direct access to IMDB.

We use the class URLConnection now. Samplecode and explainings are on this site: http://download.oracle.com/javase/tutori...iting.html

Code:

public static void testSearch() {
    BufferedReader in = null;
    try {
        URL url = new URL("http://www.imdb.de/find?s=tt&q=die+hard");
        URLConnection urlc;
        urlc = url.openConnection();
        in = new BufferedReader(new InputStreamReader(urlc.getInputStream()));
        String inputLine;

        while ((inputLine = in.readLine()) != null)
            System.out.println(inputLine);
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        try {
            in.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Again we get the response code 403. But we know that the common browsers somehow get access to it. For this reason we pretend to be Firefox:

Code:

urlc.addRequestProperty("user-agent", "Firefox");

Our method looks like this now:

Code:

public static void testSearch() {
    BufferedReader in = null;
    try {
        URL url = new URL("http://www.imdb.de/find?s=tt&q=die+hard");
        URLConnection urlc;
        urlc = url.openConnection();
        urlc.addRequestProperty("user-agent", "Firefox");
        in = new BufferedReader(new InputStreamReader(urlc.getInputStream()));
        String inputLine;

        while ((inputLine = in.readLine()) != null)
            System.out.println(inputLine);
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        try {
            in.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Et voila, it's working. The output is the source code. Try it out, if you want to see it. I won't post it since the source code is pretty much.

How do I extract information out of the source code?

Nice to have an output, but it is not as good conditioned as the IMDB-API-output. Let's have a look on a search-query with Firefox. We visit http://www.imdb.com/ and search after "die hard". There are several results. Rightclick and choose showing the source code of the site. By pressing Strg-F we look after the striking string "Popular Titles " and are able to localize it in line 499.
Alternatively you can mark the text in your browser and choose to show the selected text after rightclick.

The following shows the beginning of line 499 till the first searching result in the source code: 

Code:

<p><b>Popular Titles</b> (Displaying 5 Results)<table><tr> <td valign="top"><a href="/title/tt0095016/" onClick="(new Image()).src='/rg/find-
tiny-photo-1/title_popular/images/b.gif?link=/title/tt0095016/';"><img src="http://ia.media-imdb.com/images
/M/MV5BMTIxNTY3NjM0OV5BMl5BanBnXkFtZTcwNzg5MzY0MQ@@._V1._SY30_SX23_.jpg" width="23" height="32" border="0"></a>&nbsp;</td><td
align="right" valign="top"><img src="/images/b.gif" width="1" height="6"><br>1.</td><td valign="top"><img src="/images/b.gif" width="1" height="6">
<br><a href="/title/tt0095016/" onclick="(new Image()).src='/rg/find-title-1/title_popular/images/b.gif?link=/title/tt0095016/';">Stirb langsam</a>
(1988) <p class="find-aka">aka "Die Hard"&nbsp;- USA <em>(original title)</em></p>

For gathering information you have to look for characteristic signs of a title in the source code. Firstly the search results are in this line that contains the string "Popular Titles ". So you are able to narrow your search. Secondly you have to filter the titles itself. Those are links and bounded by <a href></a>. But you can't get them by the link-tag alone, because the corresponding pictures have links too. But we recognize, if there is an image-tag between <a href...></a> or a simple string.

Armed with this deliberations we create our first filtermethods for testing and let our program give the results for several search-queries.

Using the method contains() we find the line involving "Popular Titles ".

Code:

public void testSearch() {
    BufferedReader in = null;
    try {
        ...
        //as lons as lines are there
        while ((inputLine = in.readLine()) != null) {
            //check whether the string is in it
            if (inputLine.contains("Popular Titles")) {
                System.out.println(inputLine);
            }
        }
    } catch (IOException e) {
        ...
}

Below I introduce several methods for string processing we will use for our task.

This method looks for a character c in a string string. The search begins at index start. After finding the character, this method will return the index of it.

Code:

private Integer getIndexOfChar(String string, char c, int start) {
    Integer index = null;
    for (int i = start; i < string.length(); i++) {
        if (string.charAt(i) == c) {
            index = i;
            break;
        }
    }
    return index;
}

getSubString() uses the method above to determine a substring. The substring ranges from the character from to the character to (first occurrence is relevant).

Code:

private String getSubString(String string, char from, char to) {
    String sub;
    Integer beginIndex = getIndexOfChar(string, from, 0);
    if (beginIndex == null) {
        System.err.println("BeginIndex not found");
        return null;
    }

    Integer endIndex = getIndexOfChar(string, to, beginIndex + 1);
    if (endIndex == null) {
        System.err.println("EndIndex not found");
        return null;
    }
    sub = string.substring(beginIndex + 1, endIndex);
    return sub;
}

The method below proves whether the chararray input begins with the string compare at a given index.

Code:

private boolean containsCompareAtIndex(char[] compare, char[] input, int index) {
  //check wether the lenght of the rest input is enough
    if(compare.length > input.length - index){
        return false;
    }
  //check for inequality
    for (int j = 1; j < compare.length; j++) {
        if (compare[j] != input[++index]) {
            return false;
        }
    }
    return true;
}

getCompareIndex() returns the index from which the string input contains the string compare.

Code:

private Integer getCompareIndex(char[] compare, char[] input) {
    Integer compareIndex = null;
    for (int i = 0; i < input.length; i++) {
        if (input[i] == compare[0] && containsCompareAtIndex(compare, input, i)) {
            compareIndex = i;
            break;
        }
    }
    return compareIndex;
}

This method is our goal. getTitles() provides the list with the titles that can be found in inputLine (look up the comments for explanation).

Code:

private List<String> getTitles(String inputLine) {
    List<String> titles = new LinkedList<String>();
    char[] compare = "<a href".toCharArray();
    Integer titleBeginIndex = null;
    do {
        char[] input = inputLine.toCharArray();
        titleBeginIndex = getCompareIndex(compare, input);

        if (titleBeginIndex != null) { // probably a title was found
            //cuts inputLine till the beginning "<a href"
            inputLine = inputLine.substring(titleBeginIndex + 1);
            //determines the substring between '>' and '<'
            String title = getSubString(inputLine, '>', '<');
            //if this substring is empty, it won't be added to the list
            if (!title.equals("")) {
                titles.add(title);
            }
        }
    //go on with it until no more titles can be found
    } while (titleBeginIndex != null);
    return titles;
}

Now we should test our code. For doing this, we have to edit our testSearch() method:

Code:

public void testSearch() {
        BufferedReader in = null;
        List<String> titles = new LinkedList<String>();
        try {
            ...
            while ((inputLine = in.readLine()) != null) {
                if (inputLine.contains("Popular Titles")) {
        titles.addAll(getTitles(inputLine));
                }
            }
            for (String string : titles) {
                System.out.println(string);
            }
        } catch (IOException e) {
            ...
    }

Those are the results for "blair witch":

Quote:Blair Witch Project
Book of Shadows: Blair Witch 2

That seems to be quite good. But there are still failings. Several movies with the same title are only able to be distinguished by their year which is not filtered now. Titles with special characters are not displayed right. But for the first part it should be enough.

Deque