Showing posts with label Storage. Show all posts
Showing posts with label Storage. Show all posts

Thursday, 2 February 2017

Why in most of cases xms and XMx are same in production ?

In a production environment, if you monitor the GC data, you will notice that is a relatively short period of time (usually less than an hour), the JVM will eventually increase the heap size to the -Xmx setting. Each time the JVM increases the heap size it must ask the OS for additional memory, which takes time (and thus adds to the response time of any requests that were is process when the GC hit). And usually the JVM will never let go of that memory. Therefore, since the JVM will eventually grab the -Xmx memory, you might as well set it to that at the beginning.
Another point is that with a smaller heap size (starting with -Xms), GCs will happen more often. So by starting with a larger heap initially the GCs will happen not as often.
Finally, in a production environment, you usually run only one app server per OS (or per VM). So since the app server is not competing for memory with other apps you might as well give it the memory up front.
Note that the above is for production. It applies also to the system test environment since the system test environment should mimic production as close as possible.
For development, make -Xms and -Xmx different. Usually, you are not doing much work with the app server in development, so it will often stay with the -Xms heap setting. Also, since in development the app server will share the machine with lots of other apps (mail client, word processors, IDEs, databases, browsers, etc), setting the -Xms to a smaller size lets the app server play more nicely with the other software that is also competing for the same resources.

Data Integrity on ZFS










ZFS-Zettabyte File system

ZFS is a combined file system and logical volume manager designed by Sun Microsystems. The features of ZFS include protection against data corruption, support for high storage capacities, efficient data compression, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs.
 






Why we need to use oflag=direct while measuring disk performance using dd command

Without oflag=direct io operation uses cache/Ram because of which we will not be able to measure correct performance of a disk alone

 With oflag=direct io operations avoid using cache/Ram

LVM:Logical Volume Management

LVM stands for Logical Volume Management. It is a system of managing logical volumes, or filesystems, that is much more advanced and flexible than the traditional method of partitioning a disk into one or more segments and formatting that partition with a filesystem.
  
LVM Implementation 

 [root@highperfrawdb scripts]# ls -la /dev/xvd*
brw-rw---- 1 root disk 202, 16 Jun  8 10:16 /dev/xvdb
brw-rw---- 1 root disk 202, 17 Jun  8 10:16 /dev/xvdb1
brw-rw---- 1 root disk 202, 18 Jun  8 10:16 /dev/xvdb2
brw-rw---- 1 root disk 202, 19 Jun  8 10:16 /dev/xvdb3
brw-rw---- 1 root disk 202, 32 Jun  8 15:39 /dev/xvdc
brw-rw---- 1 root disk 202, 48 Jun  8 15:39 /dev/xvdd
brw-rw---- 1 root disk 202, 64 Jun  8 15:39 /dev/xvde
brw-rw---- 1 root disk 202, 80 Jun  8 15:39 /dev/xvdf


[root@highperfrawdb scripts]# vgcreate /dev/vgstripe /dev/xvdc /dev/xvdd /dev/xvde /dev/xvdf
  Physical volume "/dev/xvdc" successfully created
  Physical volume "/dev/xvdd" successfully created
  Physical volume "/dev/xvde" successfully created
  Physical volume "/dev/xvdf" successfully created
  Volume group "vgstripe" successfully created


[root@highperfrawdb scripts]# lvcreate -i4 -L395G -n lvstrip vgstripe
  Using default stripesize 64.00 KiB
  Logical volume "lvstrip" created


[root@highperfrawdb scripts]# lvdisplay
  --- Logical volume ---
  LV Path                /dev/vgstripe/lvstrip
  LV Name                lvstrip
  VG Name                vgstripe
  LV UUID                b8j9js-61Ib-0W8e-XUcK-z26d-eQG7-OGXjPO
  LV Write Access        read/write
  LV Creation host, time highperfrawdb, 2016-06-08 15:43:49 -0400
  LV Status              available
  # open                 0
  LV Size                395.00 GiB
  Current LE             101120
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     1024
  Block device           252:0
   


[root@highperfrawdb scripts]# mkfs -t ext4 /dev/vgstripe/lvstrip
mke2fs 1.43-WIP (20-Jun-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=16 blocks, Stripe width=64 blocks
25886720 inodes, 103546880 blocks
5177344 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
3160 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
    4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
    102400000

Allocating group tables: done                           
Writing inode tables: done                           
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done     



[root@highperfrawdb scripts]# mkdir /lvs

 
[root@highperfrawdb scripts]# mount /dev/vgstripe/lvstrip /lvs

 
[root@highperfrawdb scripts]# cd /lvs


[root@highperfrawdb lvs]# ls
lost+found



[root@highperfrawdb ~]# cat /etc/fstab

#
# /etc/fstab
# Created by anaconda on Wed Apr 15 18:13:53 2015
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=ac69bdf3-37be-47f2-9d59-6d4c672f98cc /                       ext4    defaults        1 1
UUID=947fa69d-3925-4738-a6c0-27d00f5c4f96 /boot                   ext4    defaults        1 2
UUID=d24de279-f4f1-46ae-aa3b-5297bd258dcc swap                    swap    defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
/dev/vgstripe/lvstripe      /lvs                   ext4    defaults        0 0



[root@highperfrawdb ~]# df -lh
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvdb3             25G   20G  4.5G  82% /
/dev/xvdb1            477M  148M  300M  34% /boot
/dev/mapper/vgstripe-lvstrip
                      389G   71M  369G   1% /lvs



More details 
 

RAID Array Implementation

RAID (originally redundant array of inexpensive disks, now commonly redundant array of independent disks) is a data storage virtualization technology that combines multiple physical disk drive components into a single logical unit for the purposes of data redundancy, performance improvement, or both

Different RAID Array

Tuesday, 31 January 2017

ZFS Logbias Property || Throughput vs Latency

In Oracle ZFS appliance there are two kind of disk HDD (Normal disk) and SSD (Faster I/O disk compared to HDD),
If In oracle public cloud volume is created with
logbias = throughput : data will be directly written to HDD synchronously.
logbias =latency: data will be first written to SSD (faster disk) and later will be moved to HDD in background .

Why do we need this setting and how it is useful 
SSD disk are costlier compared to HDD , therefore SSD can be used in such case where we need high throughput specially Write throughput . To optimizing cost/performance we need to have a setting to use both kind of disk