Showing posts with label Development. Show all posts
Showing posts with label Development. Show all posts

Wednesday, March 23, 2011

Finding Median in (Almost) Linear Time

I sometimes find myself needing to find the median of a set of numbers. What I would do was sort the list of numbers and then take the value in the middle index, something like(in Java):
    public static int FindMedian(List <Integer> nums)
{
Collections.sort(nums);
if(nums.size() % 2 == 0)
{
return (nums.get(nums.size() / 2) + nums.get((nums.size() / 2) + 1)) / 2;
}
return nums.get(nums.size() / 2);
}

This solution is on the order of O(n lg n) where n is the number of elements. A faster method would be use a modified quick sort. The idea being that after each partition we gain some knowledge about how many numbers are less than or equal to a given number in the list. So we can be smart and only iterate on the side of the list where we know the selected number is. Here is my implementation of QuickSelect in Java:

public class QuickSelect {
public static < T extends Comparable > T quickSelect(List < T > values, int k)
{
int left = 0;
int right = values.size() - 1;
Random rand = new Random();
while(true)
{
int partionIndex = rand.nextInt(right - left + 1) + left;
int newIndex = partition(values, left, right, partionIndex);
int q = newIndex - left + 1;
if(k == q)
{
return values.get(newIndex);
}
else if(k < q)
{
right = newIndex - 1;
}
else
{
k -= q;
left = newIndex + 1;
}
}
}
private static < T extends Comparable> int partition(List < T > values, int left, int right, int partitionIndex)
{
T partionValue = values.get(partitionIndex);
int newIndex = left;
T temp = values.get(partitionIndex);
values.set(partitionIndex, values.get(right));
values.set(right, temp);
for(int i = left; i < right; i++)
{
if(values.get(i).compareTo(partionValue) < 0)
{
temp = values.get(i);
values.set(i, values.get(newIndex));
values.set(newIndex, temp);
newIndex++;
}
}
temp = values.get(right);
values.set(right, values.get(newIndex));
values.set(newIndex, temp);
return newIndex;
}
}

From here implementing FindMedian should be pretty straight forward. The nice thing about this algorithm is that it can find any kth largest number in a set. Plus on average this will run in O(n) with a worst case of O(n lg n). Enjoy!

Saturday, December 4, 2010

in_addr to string(char*)

The other day I needed to turn a IPv4 address to a string in C. I figured some others would find it useful:


char* hexToCharIP(struct in_addr addrIP)
{
char* ip;
unsigned int intIP;
memcpy(&intIP, &addrIP,sizeof(unsigned int));
int a = (intIP >> 24) & 0xFF;
int b = (intIP >> 16) & 0xFF;
int c = (intIP >> 8) & 0xFF;
int d = intIP & 0xFF;
if((ip = (char*)malloc(16*sizeof(char))) == NULL)
{
return NULL;
}
sprintf(ip, "%d.%d.%d.%d", d,c,b,a);
return ip;
}

Tuesday, July 20, 2010

Usable Security

There has been a big push over the last few years to develop what has been coined as "usable security". Things like drawing patterns on Android devices instead of typing in a 4 digit pin or identifying particular things in an image instead of typing a password have been developed. The biggest problem with these usable security mechanisms is that they often take longer to use than the alternatives.

Imagine if you had to take your mouse and click at 10 particular spots in an image every time you wanted to unlock your screen at work. Doing this would take several more seconds at every sign on and would add up quickly. Often for systems that are used often keying in a password is still the fastest method.

Well Microsoft has developed a new solution. Instead of having password requirements that are visible to the user, like minimum length, they want to let users use anything as a password. Even simple passwords like "love" would be accepted. However there is a catch, only a small number of users will be allowed to use a particular password.

Complex password requirements were introduced to combat spraying and braying attacks. A spray and bray attack is when an attacker tries to use one particular password on a large number of accounts. This way bypassing lock out procedures. This solution by Microsoft will fix this by only allowing a small number of accounts to be compromised and thus reduce the benefits of the spray and pray attack while keeping passwords simple and easy to remember.

Thursday, July 15, 2010

Code Reviews, the Lost Art

Nearly every software development shop worth its salt has some form of what is known as a code review and nearly every developer dislikes them. Most matured developers tend to like the idea of code reviews but given the choice, on there next commit they would likely opt to not send their code for code review. The reason why is simple, code reviews can delay the forward progress of the software and they take time. When you have other developers needing access to the library you just wrote it is hard to say we need to take a few hours to a week of our time to look over my code. I think we should be able to make code reviews better and into something everyone wants to do.

After a code review I often finding myself wondering was what was found worth my time and the reviewers time? Most of the code reviews that I have been apart of have had minor suggestions or more commonly code standards compliance problems. When you rummage through several hundred or a few thousand lines of code during a code review and all that is found is that you have a few extra blank lines or should change the name of a variable, it does seem like a bit of a wast.

I'm not saying that we should not care about those extra lines or any code standard for that matter. I'm a big fan of code standards, I think they help in the readability of code. I'm saying that there is a cost to code reviews, we have to weigh those costs against the rewards. When a reviewer only finds a few compliance issues, things that could be fixed by anyone that is reading through the code, it was not worth the time the reviewer spent reviewing.

So how do we make code reviews worth everyone's time? Simple, we change the intent of a code review back to what the actual intent was. Code reviews are put into place to find bugs. Bugs that would show up to an end user or other developers that are trying to use the code.

You may say, “well Cory that is what every code reviewer is doing, they are looking for bugs.” However that is not true, sure they are looking for obvious bugs like unassigned variables being referenced, but they are not looking for deep bugs. One of the most common bugs comes from input validation, and yet it is a bug that is often over looked in code reviews. This is because it is often difficult to tell exactly where input to a function is coming from and how much it should be trusted. Detecting multi-level bugs requires a reviewer to see how the multiple levels interact and the path of the code in correct and error states. This kind of review takes a lot of time and drastically increases the complexity of a code review. The sharp increase is due to the fact that we are moving a code review from a mostly passive practice to a very active process.

Obviously code reviews could not detect all bugs and there will be times when a code review will not find any bugs. From this active process of a code review you get a new found level of confidence. Bug counts will be decrease and actual development should increase. This confidence is how a code review pay for themselves.

Thursday, July 8, 2010

OpenVPN in Windows 7

I often need to work remotely from work, luckily my work has a VPN server that allows me to get access to the companies internal resources. I have been using OpenVPN in Windows XP for a long time to do this, through the use of OpenVPN GUI. Well when I got a new laptop it came with Windows 7 installed. So one of the first things I did was set up my development environment which required me to get into some of the file shares inside of my companies network. I thought it wouldn't be a problem at all to do, I installed OpenVPN GUI and just copied over my configuration and key files. When I went to connect I got quite an interesting error:

Thu Jul 08 23:05:33 2010 ROUTE: route addition failed using CreateIpForwardEntry: One or more arguments are not correct.   [if_index=16]

It turns out to be very simple to fix. In Windows 7 and I believe in Vista you need to do a few extra steps to get OpenVPN GUI to work with Windows 7. First you need to go C:\Program Files\OpenVPN\Bin and make sure openvpn-gui.exe is always started as Administrator (in the compatibility menu of the file properties). Then you will need to edit your configuration file and add two lines after the line that describes your cipher:



route-method exe

route-delay 2


That should do it. Let me know if you have any questions.

Tuesday, July 6, 2010

Nexus One Support in Ubuntu 10.04

I recently purchased the Nexus One from Google. It is quite an amazing phone and I've been really happy with it. I've taken interest in developing for the Android platform now that I have one. Luckily Google has provided a very good resource for Android app development at http://developer.android.com/.

Today I was working on allowing me to upload my applications onto my Nexus One so that I could test the software on a real device. Well I ran into an issue after following Google's tutorial on how to do it located here. I run Ubuntu 10.04 on my main development machine and so I followed the instructs for how to do it for Linux. Well when I got to the end of the tutorial when it tells you to test your the setup with the following command:
adb devices

I got the following output:

List of devices attached

???????????? no permissions

It turns out that Google's instructions have not been updated so reflect how to get the Nexus One to work under Linux. Well after a bit of research I came across the following blog post that explained it. Basically the list of USB Vendor IDs does not have listed the proper value for the Nexus One. I had selected to use HTC's Vendor ID ("0bb4") being that the Nexus One is actually produced by HTC, however that is incorrect. Instead you need to use the Vendor ID of "18d1". So the following line should be in /etc/udev/rules.d/51-android.rules


SUBSYSTEM=="usb", SYSFS{idVendor}=="18d1", MODE="0666"

NOTE: Be sure not to capitalize the D in 18d1, doing this will cause this solution not to work.

Now that you have added that line to the file, you will simply need to restart udev. With the Nexus One unplugged from your system run this command:



sudo service udev restart


Now you can plug in your Nexus One and run:


adb devices

You should see something like this:


List of devices attached

???????????????? device

I hope this helps and post below if you need any further help.

Sunday, May 2, 2010

Examining Security Of Open Source and Closed Source

I recently just completed a research paper, with two of my colleagues: Clint Caywood and Matt Strayhall, on the security of Open Source Software. The paper went very in depth and I feel helped fill a void of the lack of credible information in this hotly debated topic. Here is the abstract:
In this paper, we examine the security of open source software versus that of closed source software. Facets examined include a brief history of the growing need for security in software, a comparison of the different philosophies driving the development of security in open and closed source software, arguments for obscurity in closed source versus the “many eyeballs” theory in open source, and the pros and cons involved with both development processes. We also look at the two approaches in practice, focusing on competing software like Linux and Windows, OpenOffice.org and Microsoft Office, and Apache and Windows IIS Server. Finally, we examine the impacts on society from software security, as well as who is responsible for maintaining secure software.

You can find more, including download links, if you visit my Research section of this website.

Floating Point Guide

I came across a great article about the pit falls of using floating point numbers in programming. All to often software bugs come up because developers expect something like 0.1+0.2==0.3. The article goes into very clear detail as to why it is not the case that we can expect exact equality while using float point representations.

Tuesday, December 15, 2009

Fast Case Converter

Often I find myself needing to convert between upper case to lower case and vice versa. This is a fairly simple thing to do if you understand ASCII encoding. A good reference for the ASCII Table can be found here. As you can see, to convert a letter to lower case you would add 32 to the current value and subtract 32 to convert it from lower to upper. However to implement this it takes a few lines of code to put some conditions in that check to see the current case and decide to add or subtract.

I was looking at the ASCII Table the other day and came up with a simplistic method to swap the case of a given character. That is to exclusively-or the value of the character with 0x20. This will convert the upper case to lower and the lower to upper. Its probably the fastest method to swap ,CPUs can xor in one cycle (excluding memory load time). However you still need some type of conditions to make sure the character is actually a letter and not a number of symbol.

Enjoy!

Tuesday, October 27, 2009

MySQL and Java common problem

Have you ever received the following error while using Java and MySQL:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '????????????????' at line 1

This is a common error when you first set up Java and MySQL to work together. To correct it simply add the following lines of code to your my.cnf in the section [mysqld]
collation_server=utf8_general_ci
character_set_server=utf8

Sunday, October 18, 2009

Self Replicating Code

For a class recently the teacher challenged us to write a code snip-bit that would be self replicating. By that I mean write a program that when executed would write out its own source code. This is a bit harder than you'd initially think it to be. My first attempt tried to store the source code within the program when execute it would output what was stored within. However this creates a chick and egg problem. The source code contains the source code. Something that at first stumped me. Then I remembered Stringification in the C preprocessor and I came up with this:

#include "stdio.h"
#define M(Code) main(){FILE* fp = fopen("source.c", "w");fprintf(fp,"#include \"stdio.h\"\n#define M(Code) %s\nM(%s)\n",#Code,#Code);fclose(fp);}
M(main(){FILE* fp = fopen("source.c", "w");fprintf(fp,"#include \"stdio.h\"\n#define M(Code) %s\nM(%s)\n",#Code,#Code);fclose(fp);})


This is a pretty cute program if I can say so myself. When its ran it will create a file called source.c that will be exactly the same as the code above. You could even recompile source.c and it will create another source.c. You should attempt to do this on your own to see if you can come up with other cool ways of doing it.

Download: SelfRep.c

Thursday, August 13, 2009

Twitter can be neat

If  you are like me you don't exactly see the usefulness of the social networking site Twitter. After reading a recent wired article I can not better see that usefulness. There appears to be some pretty clever people connecting common house hold devices to Twitter.

This really isn't all that new of a concept however. People have been connecting bots to IRC for control for a very long time now. I would suspect similar devices have been plugged into IRC. I wonder about the secuirty behind putting so much control of your house onto the net. Losing control of your own account could give an attacker control of say your air conditioning and drop your house to 60 degrees. Causing your electric bill to ski rocket.

Saturday, August 8, 2009

Great C Tutorial

Dr. Dave Marshall at Cardiff School of Computer Science has written a great C tutorial. This tutorial is mostly aimed at UNIX C developers and covers the basics of C programming to advanced multi-threaded application development. This is the tutorial that I used to learn C and I still open it up whenever I need to look up something new or need a quick refresher on something. Unfortunately it appears that Google keeps burying the link so I thought it would be helpful to others that are looking for in my opinion the best C tutorial available online.