C# Convert Int to Byte Array

An Integer in C# is stored using 4 bytes with the values ranging from -2,147,483,648 to 2,147,483,647. Use the BitConverter.GetBytes() method to convert an integer to a byte array of size 4.

One thing to keep in mind is the endianness of the output. BitConverter.GetBytes returns the bytes in the same endian format as the system. This is most likely little-endian in your case. If you need the output in big-endian format (which is the standard as per RFC1014 3.2) the output byte array needs to be reversed using Array.Reverse().

Here’s the portable version of the entire code that checks the endianness of the system and always returns the byte array in Big-Endian format.

1
2
3
4
int number;
byte[] bytes = BitConverter.GetBytes(number);
if (BitConverter.IsLittleEndian)
    Array.Reverse(intBytes);
Tagged with:

C# Decode URL

There are several ways you can decode URLs in C#. It all depends on what framework you are using.

.NET Core

If you are using .NET Core (either ASP .NET, a Class library or a Console App) or even .NET Standard, you can use one of these two methods:-

System.Web.HttpUtility.UrlDecode()

OR

System.Net.WebUtility.UrlDecode()

ASP .NET Framework

If you are inside an instance method of System.Web.Mvc.Controller, you can use the Server property as follows:-
Server.UrlDecode()

For any other class, you can use one of these:-
System.Web.HttpUtility.UrlDecode()

OR

System.Net.WebUtility.UrlDecode()

.NET Framework Console/Desktop Application

For the remaining portions of the .NET platform like a .NET Framework Console Application, Class Library, Windows Service & Desktop Application you can use the following:-
System.Net.WebUtility.UrlDecode()

On a side note, you can try my online URL Encoder/Decoder.

Code Signing Certificates – Why/When to Use

What are code signing certificates?

Code Signing Certificates are used to digitally sign binaries (Executables and DLLs).

Why to sign binaries?

Signing the binaries ensure that the files are from a trusted source (you/your company) and that they have not been tampered by someone else.

What to sign?

You should use them if you are distributing binaries that you built to a customer. These include the executable (EXE) for your application and any libraries (DLLs) you built to modularize the application. Chances are, that you are also creating a windows installer to package your application. You should sign it too.

What do I need for signing?

  • Your binaries
  • A code signing certificate
  • A code signing tool

Timestamps Quirks

Just like an SSL certificate, your code signing certificate has a validity period. If you forget to renew your SSL certificate, browsers will not allow the users to get into your site. Well, unless they are really desperate and bypass the protection. In which case, I would like to know what content you have up there! Thankfully, the certificate providers keep spamming you about the impending expiry and you get a new certificate, put it on your server and every thing is hunky-dory.

The same approach however will not work for code signing. You do not put the certificate/private key with your application. You use it to sign the binaries and embed that information in the binaries itself (it actually modifies your binary). Your certificate might expire 2 years from now, but the binaries must keep on working beyond that. To do this, we utilize a Timestamp server from a trusted authority. This is done during the code signing phase. The signing tool hits the timestamp server and embeds the information in your binary. Congratulations, your application is now Omnitemporal! Operating systems will never warn the user that the code signing certificate has expired, even after the actual expiry date of the code signing certificate has long gone.

Creating and Applying SSL Certificates (Complete Procedure)

I am writing this post in hope that it will help others by saving hours of research in trying to generate and use SSL certificates.

These steps outline the complete activity needed for generating SSL certificates for a web server in Java. For clarity, I have also included the steps that are done by the Certificate Authority. As such, you can follow these steps exactly and experience the role each of them plays.

Activity 1: Creating the Root CA

Step 1: Create a Private Key for the Root CA

openssl genrsa -out ca.key 4096

Step 2: Create the Self-Signed Certificate of the Root CA

openssl req -sha256 -new -x509 -days 1826 -key ca.key -out ca.crt

Root CA Certificates are always self signed. These certificates are valid for a long period, like 20-25 years.

Notice the -sha256 option, it forces the certificate to use the now mandatory SHA-2 instead of the unsecure SHA-1 algorithm which is blocked by browsers

Activity 2: Creating the Intermediate Certificate

Step 1: Create a Private Key for the Intermediate CA

openssl genrsa -out ia.key 4096

Step 2: Generate a Certificate Signing Request (CSR) for the Intermediate CA Certificate

openssl req -new -key ia.key -out ia.csr

Unlike the Root CA Certificate, this one cannot be self signed. It will be signed by the Root CA. The CSR created during this step is used by the Root CA to generate this certificate.

Step 3: Create an extension file for the Intermediate CA Certificate

echo "basicConstraints=CA:TRUE" > ia.ext

This step is needed to give the Intermediate CA the authority to generate certificates for others. If this step is omitted along with the -extfile option in the next step, the browser will display a warning saying that the Intermediate CA is not authorized to generate certificates.

This rule does not apply to the Root CA certificates. It is needed to differentiate between End Entity and Intermediate CA certificates. Without this security in place, all end entity certificates (like for your server) would have been able to generate certificates for others while inheriting the trust from the Root CA.

Step 4: Create the Intermediate CA Certificate signed by the Root CA

openssl x509 -req -sha256 -days 730 -in ia.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out ia.crt -extfile ia.ext

Notice the -extfile option which passes the extensions file from the previous step.

Activity 3: Creating the Server Certificate for your Web Server

Step 1: Create a Private Key for the Server

openssl genrsa -out test.maxotek.net.key 4096

Step 2: Generate a Certificate Signing Request (CSR) for the Server Certificate

openssl req -new -key test.maxotek.net.key -out test.maxotek.net.csr

In this case, the Intermediate CA will generate the certificate for our server. So, a CSR needs to be generated and sent to the Intermediate CA.

Step 3: Create the Server Certificate signed by the Intermediate CA

openssl x509 -req -sha256 -days 730 -in test.maxotek.net.csr -CA ia.crt -CAkey ia.key -set_serial 02 -out test.maxotek.net.crt

We don’t need the -extfile option or the extensions file in this case. This is because this Server certificate is an End Entity certificate and it should not have the authority to generate certificates.

Activity 4: Using the certificates with JAVA

If you did not use keytool to create the private key and CSR (like in our case), there is a bit of a trick to using these certificates with JAVA. Keytool does not allow you to import the private key directly. This is for security reasons because the private key should never leave the server. You don’t need to provide your private key to the CA for it to generate a certificate for you.

Step 1: Chain Root and Intermediate Certificates

cat ia.crt ca.crt > bundledca.crt

This is very important because the client will only have the Root CA’s certificate in it’s certificate store. All intermediate certificates must be passed by the server to the client.

Step 2: Convert the certificate and private key to the Intermediate PKCS12 format

As keytool lacks (more like intentionally omitted) the ability to import the private key directly, we are going to use the PKCS12 intermediate format. This PKCS12 file will have the private key of our server along with the public keys of our server, the Intermediate CA and the Root CA chained together.

openssl pkcs12 -export -in test.maxotek.net.crt -inkey test.maxotek.net.key -out test.maxotek.net.p12 -name testmaxotek -CAfile bundledca.crt -caname gidia -chain

Step 3: Convert the PKCS12 file to Java Keystore

Finally we convert the PKCS12 file to JAVA’s Keystore format which can then be used by a web server like Tomcat, JBoss.

keytool -importkeystore -deststorepass changeit -destkeypass changeit -destkeystore test.maxotek.net.keystore -srckeystore test.maxotek.net.p12 -srcstoretype PKCS12 -srcstorepass changeit -alias testmaxotek

The important thing here is to match the alias with the one used in the previous step.

Useful Commands

Convert Keystore to PKCS12

keytool -importkeystore -srckeystore test-142.keystore -destkeystore test-142.p12 -deststoretype PKCS12

Export the certificate

openssl pkcs12 -in test-142.p12 -nokeys -out test-142.pem

Export the private key

openssl pkcs12 -in test-142.p12  -nodes -nocerts -out test-142.key

Extracting data from Web Pages

Considering the volume of data available in the World Wide Web today, it is a no brainer that people will often need to extract data from it. This is the first part of a series of posts that will show you how to extract data from Web Pages using Data Utensil.

For those of you who are new here, Data Utensil is our product which aims to be “The single tool for all your data needs”. Now that is a big goal, but we are working towards it, albeit in tiny steps. As of writing this article, you can explore & manage databases, compare schemas & data, import & export tabular data from various formats and crawl websites using Data Utensil. I will stop my bantering there and get right on the topic.

Extracting data from web pages comprises of two main activities, crawling web pages & importing the data from HTML markup. Data Utensil splits these two activities in different Jobs. A job is a long running activity which runs in the background. Different types of jobs accomplish different things. For instance, the data comparison job compares data in tables of two schemas while the copy schema job copies data of tables in one schema to another.

The Crawl Website job crawls web pages and dumps the resultant HTML as files. Crawling starts with one or more URLs from where Data Utensil discovers new URLs to crawl, all the time dumping the HTML markup of the web pages. You can specify path filters to exclude URLs from being crawled or include path filters to restrict the crawling to only those. All the HTML dumps are saved in a folder of your choice. The dumps are stored in folders and files that try to mimic the path hierarchy of the URL. So, the following page: http://maxotek.com/products/data_utensil will be saved to C:\My Crawls\maxotek.com\products\data_utensil.html

After the completion of crawling you end up with a bunch of HTML files. These serve as the input to the next step, Import Table from HTML. There can be multiple types of tabular datasets in these files. To choose the correct one, you can select a file from the dump & see a preview of the tabular datasets in this file. You can easily switch to other tabular datasets contained in the file, to locate the correct one. The software understands the table through it’s XPath and column names. Data from all tables matching these two criteria will be appended to a new table. In addition to the columns in the table, you can specify virtual columns which can extract data from any HTML node/attribute from the dump file using XPaths. Virtual columns can also extract the name of the HTML file or any folder in it’s path.

After completing the configuration of columns & virtual columns, you can choose the schema & specify a name for the new database table which will be created. Finally, you specify a Name & Description for this Job.

When you run the job, it will use these configurations and start importing the data from HTML markup.

I am working on a new type of Job that can automatically extract multiple datasets from HTML dumps. The next article will focus on how this automation can save time, while accomplishing the same results.

View & Edit Table Definition

This post briefly explains the different views available for Managing a Table’s definition using Data Utensil.

This is where you can view the columns, indexes and foreign keys of a Table.

Table definition is comprised of several views:-

Structure

Table Structure

This view shows the columns of a table and their properties such as:-

  • Data Type – The datatype of the column.
  • Size – The size of the column (if applicable)
  • Precision – The precision of the column (if applicable)
  • Scale – The scale of the column (if applicable)
  • Primary Key – Whether or not the column is part of the Primary Key
  • Nullable – Whether or not the column can store NULL values
  • Unique – Whether the values of the column must be unique
  • Identity – Whether it is an Identity column of SQL Server whose values are automatically generated by the database on insertion
  • Collation – The collation of the column

Indexes

Indexes

Indexes for this table along with their properties are listed here.

  • Type – The type of the Index – Clustered, Non Clustered, etc
  • Name – The name of the Index
  • Columns – The columns on which the Index applies
  • Unique – Whether it is a Unique Index
  • Primary Key – Whether it is a Primary Key Index

Foreign Keys

Foreign Keys

Foreign keys of the table and their properties are found here.

  • Constraint Name – The name of the foreign key constraint
  • Referenced Table – The referenced (Master) table of the foreign key
  • Referencing Table – The referencing (Child) table of the foreign key
  • Referenced Columns – The columns in the master table which are part of the foreign key
  • Referencing Columns – The columns in the child table which are part of the foreign key
  • On Update – The ON UPDATE rule for the foreign key
  • On Delete – THe ON DELETE rule for the foreign key

SQL Definition

Table Definition

This view shows the SQL query that can be used to create the Table.

Sql Preview

SQL Preview

As you make changes to an existing table or add columns, indexes and foreign keys to a new table, the SQL that will be fired to make the changes are shown in this view.

Table Properties

This view shows the basic properties for the table such as its’ Name, Schema, etc.