[Book Review] Visual Quickstart Guide – Unix and Linux

I need to refresh my knowledge about basic commands in Linux. I browse some reference books but I do not want to read detailed information. So, I try to find some linux pocket books. I get ‘Visual Quickstart Guide – Unix and Linux (Fifth Edition)‘ by Deborah S. Ray and Eric J. Ray.

Overall, yeah.. it is not for advanced users or one who is looking detailed information. It lists only common commands that we can use in daily administration tasks. Those are:

How to work with Directories and Files (e.g. ls, mkdir, find, touch, tar, gzip, etc)

How to manipulate files and control their ownership and permissions

How to write basic scripts

How to edit files using vim and nano

How to sending and reading emails

How to access internet (e.g. using ssh, telnet, ftp, etc)

How to get information about the system

 
Beyond that, this book also provides references about commands, flags and arguments. Those can be used as your handy utilities when you do your tasks.

Common Unix/Linux directories

Common Unix/Linux directories and their contents.

Directory	Contents
/bin	Essential programs and commands for use by all users
/boot	Files used in booting the operating system
/dev	Device folder, such as alarm, cdrom, cpu, cdrw, etc.
/etc	System configuration files and global settings
/etc/sysconfig	Files that control the system configuration
/etc/X11	Configuration files for X Window System
/lib	Shared libraries
/home	Home directories for users
/lost+found	Recovering files which are not properly closed due to many reason such as power failure
/misc	This directory is used for automatically mounting directories on removable devices and remote directories
/mnt	The mount points for file systems mounted after the system is booted
/opt	Optional application software packages
/proc	Location of virtual file system
/root	Home directory for root user
/sbin	Essential system binaries
/tmp	Temporary files
/usr/bin	Commands and programs that are less central to basic Unix system functionality than those in /bin but still useful and often important
/usr/include	Standard include file and header files for C programs
/usr/lib	Libraries for installed packages
/usr/local	Most files and data that were developed or customized on the system
/usr/share	Shared data file
/var	Changeable data, including system logs, temporary data from programs, and user main storage

The power of User-Agent

My good friend, Mike Wazowski, called and asked my help. He told me that he encountered something strange when he looked for a tour package at a travel agency website. His family had a plan to go to Europe. He and his wife searched some travel agency website. They browsed same site and got different prices for same tour. He asked me to investigate this issue.

First, I tried to do what he did and got same price as his wife got. Then, I asked him to show me how he did. Surprisingly, it was true that he got more expensive price for same one. I opened my BurpSuite and set it as a proxy to analyze HTTP traffics. Then, I did some hacking techniques against that travel agency website.

Finally, I found that it is because of HTTP Header User-Agent. That travel agency website filtered its customer based on value of User Agent. It charged more expensive to my friend because he use Apple laptop. I did an experiment to intercept his HTTP Request and change its User Agent.. and… I got normal price.

What is User-Agent? How do we manipulate it?

User-Agent is our representative in communication within a client-server system. Especially in HTTP, it identify the software we use to send our request. So, our browser populates this values automatically when we surf in the Internet.

Most Web browsers use a User-Agent with this format:
Mozilla/[version] ([system and browser information]) [platform] ([platform details]) [extensions]

For example, Safari on the iPad has used the following:

Mozilla/5.0 (iPad; U; CPU OS 3_2_1 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Mobile/7B405

Usually, User-Agent is used for:

• To detect browser version so that application at server side can decide whether it support this particular browser and version.
• To detect client type (e.g. mobile, PC, wearable device, etc) so that application at server side can response a layout depends on its type.
• Advertise companies use it for statistic purpose.

In Mike’s case, the application at server side filters its customer based on his platform. Hahaha.. smart.

Let’s see from the other side… at Black Hat Hacker side 🙂

User-Agent is same as other input fields. It can be used to attack the application at server side. If User-Agent value is not validated and sanitized… a hacker can exploit the system.

What are those attacks can be done by a hacker?

a. Stored and Reflected Cross Site Scripting (XSS)

User-Agent is the only HTTP Request header as a prime candidate  for XSS. The hacker modifies User-Agent values to become like this

User-agent: Mozilla/5.0 alert(‘XSS’);<!–

Of course, it is not as simple as that… the hacker will write a more evil script.

 

b. SQL Injection

A large number of web analytic tools store User-Agent value in database. Imagine that User-Agent is not validated and a hacker inject with sql query. For example

User-­‐agent:       Mozilla/5.0       (iPad;       U;       CPU       OS       3_2       like       Mac       OS       X;       en-­‐us)       AppleWebKit/532.21.10       (KHTML,       like       Gecko)       Version/4.0.4       Mobile/7C445c    
Safari/531.21.102011-­‐10-­‐16       20:23:50’

Note the single quote at the end of value.

 

c. Denial of Service

It can be happen when a hacker fill User-Agent with characters whose length is more than 255 characters. For more detail, you can read CVE-2004-0169.

 

Now we can see that we can manipulate User-Agent easily. It can be from both client or server sides. You must stay vigilant. You can refer to this site, http://www.useragentstring.com/, if you want to analyze unusual User-Agent value.

Are you sure that you are using secured third party libraries?

Are you sure that you are using secured third party libraries?

Hmm… another question… Does big name always bring secure product? I can answer.. maybe yes.. maybe no.

However, for sure, you must do your own research before putting third party libraries into your application. I give you one example. Your application processes sensitive information and put those in a database. You use third party libraries to retrieve data before doing some other processing in your logic. How do we make sure that those third party libraries are secured enough?
Remember, those libraries are transferring our sensitive information. It is quite risky if we do not find any information about it.

One way that you can do is searching in vulnerability databases and tracking systems. Now, there are many systems provide information about product vulnerabilities.

The National Vulnerability Database (NVD)

NVD is the U.S. government repository of standards based vulnerability management data represented using the Security Content Automation Protocol (SCAP). This data enables automation of vulnerability management, security measurement, and compliance. NVD includes databases of security checklists, security related software flaws, misconfigurations, product names, and impact metrics.

US Computer Emergency Response Team (CERT) Vulnerability Notes Database

The CERT Knowledgebase is a collection of internet security information related to incidents and vulnerabilities.

Open Source Vulnerability Database

It is an independent and open-sourced database that provides accurate, detailed, current, and unbiased technical information on security vulnerabilities.

Common Vulnerabilities and Exposures (CVE)

Common Vulnerabilities and Exposures (CVE®) is a dictionary of common names (i.e., CVE Identifiers) for publicly known information security vulnerabilities.

Common Weakness Enumeration

It is targeted to developers and security practitioners, the Common Weakness Enumeration (CWE) is a formal list of software weakness types created to:
a. Serve as a common language for describing software security weaknesses in architecture, design, or code.
b. Serve as a standard measuring stick for software security tools targeting these weaknesses.
c. Provide a common baseline standard for weakness identification, mitigation, and prevention efforts.

Default Ports

This is a list of default (and popular) ports. It will be useful when you are doing port scanning.

Port	Description
20, 21	File Transfer Protocol
22	SSH (Secured Shell)
23	Telnet
25	Simple Mail Transfer Protocol
53	Domain Name Service
69	Trivial File Transfer Protocol
70	Gopher
80	HTTP
88	Kerberos
110	Post Office Protocol 3
115	Secured FTP
119	Network News Transfer Protocol
137,138	NetBIOS Datagram Service (UDP)
139	NetBIOS Session Service (TCP)
143	Internet Message Access Protocol
160,161,162	Simple Network Management Protocol
389	LDAP Server (TCP/UDP)
443	HTTPS
636	Secure LDAP Server
1433	Microsoft SQL listening port
1701	Layer 2 Tunneling Protocol. Used to establish VPN connections
1723	Point-to-point Tunneling Protocol. Used to establish VPN connections
1433	Microsoft SQL listening port
3306	My SQL
3389	Remote Desktop Protocol
8080	Glassfish Java Application Server

HTTP/1.1 Status Code

HTTP/1.1 status codes use three-digit integer as response for each HTTP request. The first digit of the status code refers to the class of response. There are five values for the first digit:

1xx: Informational – Request received and continuing process

2xx: Success – The action was successfully received and accepted

3xx: Redirection – Further action must be taken in order to complete the request

4xx: Client Error – The request contains bad syntax or cannot be fulfilled

5xx: Server Error – The server failed to fulfill an apparently valid request

Response class	Status Code	Description
1xx: Informational	100	Continue
	101	Switching protocols
2xx: Success	200	OK
	201	Created
	202	Accepted
	203	Non-Authoritative
	204	No Content
	205	Reset Content
	206	Partial Content
3xx: Redirection	300	Multiple Choices
	301	Moved Permanently
	302	Found
	303	See Other
	304	Not Modified
	305	Use Proxy
	307	Temporary Redirect
4xx: Client Error	400	Bad Request
	401	Unauthorized
	402	Payment Required
	403	Forbidden
	404	Not found
	405	Method Not Allowed
	406	Not Acceptable
	407	Proxy Authentication Required
	408	Request Time-out
	409	Conflict
	410	Gone
	411	Length Required
	412	Precondition Failed
	413	Request Entity Too Large
	414	URI Too Long
	415	Unsupported Media Type
	416	Request Range not satisfiable
	417	Expectation Failed
5xx: Server Error	500	Internal Server Error
	501	Not Implemented
	502	Bad Gateway
	503	Service Unavailable
	504	Gateway Time-out
	505	HTTP Version not supported

HTML URL Encoding Reference

URL Encoding is used to convert characters into a format that can be transmitted over the internet.

These are URL Encoding for character-set (by default it is UTF-8 in HTML5)

Character	From Windows-1252	From UTF-8
space	%20	%20
!	%21	%21
“	%22	%22
#	%23	%23
$	%24	%24
%	%25	%25
&	%26	%26
‘	%27	%27
(	%28	%28
)	%29	%29
*	%2A	%2A
+	%2B	%2B
,	%2C	%2C
–	%2D	%2D
.	%2E	%2E
/	%2F	%2F
0	%30	%30
1	%31	%31
2	%32	%32
3	%33	%33
4	%34	%34
5	%35	%35
6	%36	%36
7	%37	%37
8	%38	%38
9	%39	%39
:	%3A	%3A
;	%3B	%3B
<	%3C	%3C
=	%3D	%3D
>	%3E	%3E
?	%3F	%3F
@	%40	%40
A	%41	%41
B	%42	%42
C	%43	%43
D	%44	%44
E	%45	%45
F	%46	%46
G	%47	%47
H	%48	%48
I	%49	%49
J	%4A	%4A
K	%4B	%4B
L	%4C	%4C
M	%4D	%4D
N	%4E	%4E
O	%4F	%4F
P	%50	%50
Q	%51	%51
R	%52	%52
S	%53	%53
T	%54	%54
U	%55	%55
V	%56	%56
W	%57	%57
X	%58	%58
Y	%59	%59
Z	%5A	%5A
[	%5B	%5B
\	%5C	%5C
]	%5D	%5D
^	%5E	%5E
_	%5F	%5F
`	%60	%60
a	%61	%61
b	%62	%62
c	%63	%63
d	%64	%64
e	%65	%65
f	%66	%66
g	%67	%67
h	%68	%68
i	%69	%69
j	%6A	%6A
k	%6B	%6B
l	%6C	%6C
m	%6D	%6D
n	%6E	%6E
o	%6F	%6F
p	%70	%70
q	%71	%71
r	%72	%72
s	%73	%73
t	%74	%74
u	%75	%75
v	%76	%76
w	%77	%77
x	%78	%78
y	%79	%79
z	%7A	%7A
{	%7B	%7B
|	%7C	%7C
}	%7D	%7D
~	%7E	%7E
	%7F	%7F
`	%80	%E2%82%AC
	%81	%81
‚	%82	%E2%80%9A
ƒ	%83	%C6%92
„	%84	%E2%80%9E
…	%85	%E2%80%A6
†	%86	%E2%80%A0
‡	%87	%E2%80%A1
ˆ	%88	%CB%86
‰	%89	%E2%80%B0
Š	%8A	%C5%A0
‹	%8B	%E2%80%B9
Œ	%8C	%C5%92
	%8D	%C5%8D
Ž	%8E	%C5%BD
	%8F	%8F
	%90	%C2%90
‘	%91	%E2%80%98
’	%92	%E2%80%99
“	%93	%E2%80%9C
”	%94	%E2%80%9D
•	%95	%E2%80%A2
–	%96	%E2%80%93
—	%97	%E2%80%94
˜	%98	%CB%9C
™	%99	%E2%84
š	%9A	%C5%A1
›	%9B	%E2%80
œ	%9C	%C5%93
	%9D	%9D
ž	%9E	%C5%BE
Ÿ	%9F	%C5%B8
	%A0	%C2%A0
¡	%A1	%C2%A1
¢	%A2	%C2%A2
£	%A3	%C2%A3
¤	%A4	%C2%A4
¥	%A5	%C2%A5
¦	%A6	%C2%A6
§	%A7	%C2%A7
¨	%A8	%C2%A8
©	%A9	%C2%A9
ª	%AA	%C2%AA
«	%AB	%C2%AB
¬	%AC	%C2%AC
­	%AD	%C2%AD
®	%AE	%C2%AE
¯	%AF	%C2%AF
°	%B0	%C2%B0
±	%B1	%C2%B1
²	%B2	%C2%B2
³	%B3	%C2%B3
´	%B4	%C2%B4
µ	%B5	%C2%B5
¶	%B6	%C2%B6
·	%B7	%C2%B7
¸	%B8	%C2%B8
¹	%B9	%C2%B9
º	%BA	%C2%BA
»	%BB	%C2%BB
¼	%BC	%C2%BC
½	%BD	%C2%BD
¾	%BE	%C2%BE
¿	%BF	%C2%BF
À	%C0	%C3%80
Á	%C1	%C3%81
Â	%C2	%C3%82
Ã	%C3	%C3%83
Ä	%C4	%C3%84
Å	%C5	%C3%85
Æ	%C6	%C3%86
Ç	%C7	%C3%87
È	%C8	%C3%88
É	%C9	%C3%89
Ê	%CA	%C3%8A
Ë	%CB	%C3%8B
Ì	%CC	%C3%8C
Í	%CD	%C3%8D
Î	%CE	%C3%8E
Ï	%CF	%C3%8F
Ð	%D0	%C3%90
Ñ	%D1	%C3%91
Ò	%D2	%C3%92
Ó	%D3	%C3%93
Ô	%D4	%C3%94
Õ	%D5	%C3%95
Ö	%D6	%C3%96
×	%D7	%C3%97
Ø	%D8	%C3%98
Ù	%D9	%C3%99
Ú	%DA	%C3%9A
Û	%DB	%C3%9B
Ü	%DC	%C3%9C
Ý	%DD	%C3%9D
Þ	%DE	%C3%9E
ß	%DF	%C3%9F
à	%E0	%C3%A0
á	%E1	%C3%A1
â	%E2	%C3%A2
ã	%E3	%C3%A3
ä	%E4	%C3%A4
å	%E5	%C3%A5
æ	%E6	%C3%A6
ç	%E7	%C3%A7
è	%E8	%C3%A8
é	%E9	%C3%A9
ê	%EA	%C3%AA
ë	%EB	%C3%AB
ì	%EC	%C3%AC
í	%ED	%C3%AD
î	%EE	%C3%AE
ï	%EF	%C3%AF
ð	%F0	%C3%B0
ñ	%F1	%C3%B1
ò	%F2	%C3%B2
ó	%F3	%C3%B3
ô	%F4	%C3%B4
õ	%F5	%C3%B5
ö	%F6	%C3%B6
÷	%F7	%C3%B7
ø	%F8	%C3%B8
ù	%F9	%C3%B9
ú	%FA	%C3%BA
û	%FB	%C3%BB
ü	%FC	%C3%BC
ý	%FD	%C3%BD
þ	%FE	%C3%BE
ÿ	%FF	%C3%BF

These are URL Encoding for control characters

ASCII Character	Description	URL-encoding
NUL	null character	%00
SOH	start of header	%01
STX	start of text	%02
ETX	end of text	%03
EOT	end of transmission	%04
ENQ	enquiry	%05
ACK	acknowledge	%06
BEL	bell (ring)	%07
BS	backspace	%08
HT	horizontal tab	%09
LF	line feed	%0A
VT	vertical tab	%0B
FF	form feed	%0C
CR	carriage return	%0D
SO	shift out	%0E
SI	shift in	%0F
DLE	data link escape	%10
DC1	device control 1	%11
DC2	device control 2	%12
DC3	device control 3	%13
DC4	device control 4	%14
NAK	negative acknowledge	%15
SYN	synchronize	%16
ETB	end transmission block	%17
CAN	cancel	%18
EM	end of medium	%19
SUB	substitute	%1A
ESC	escape	%1B
FS	file separator	%1C
GS	group separator	%1D
RS	record separator	%1E
US	unit separator	%1F

Source :  http://www.w3schools.com/tags/ref_urlencode.asp