Technical troubleshooting and more

While trying to start one of Vertica nodes you may face a data consistency problem. from vertica.log <PANIC> @v_vdb_node0005: VX001/2973: Data consistency problems found; startup aborted         HINT:  Check that all file systems are properly mounted.  Also, the --force option can be used to delete corrupted data and recover from the cluster         LOCATION:  mainEntryPoint, /scratch_a/release/svrtar5575/vbuild/vertica/Basics/vertica.cpp:1613 so.. don't PANIC :-) solution: restart the problematic node with force flag which will repair the corruptions from buddy nodes.  [dbadmin ]$ /opt/vertica/bin/admintools -t restart_node -d $db_name -s $host --force and the result: *** Restarting nodes for database vdb ***         restart host node0005 with catalog v_vdb_node0005_catalog         issuing multi-node restart         Start...

if you face this kind of error with your remote NFS: [root@Vertica000 ~]# mount /files/application/Rremote3 mount.nfs: backgrounding "10.0.0.2:/files/application/remoteFiles" mount.nfs: mount options: "bg,hard,nointr,rsize=65536,wsize=65536,tcp,actimeo=0,vers=3,timeo=600,addr=10.0.0.2" look for the problem in the log file: cat /var/log/messages | grep mount  mount to NFS server '10.0.0.2' failed: timed out, retrying Solution: In most of the cases, you have a problem with your iptables in the destination server login as root to dest server (10.0.0.2) in my case and type this command: iptables --flush  the go back to your origin server to try remount the problematic NFS file system of course this is in case nfs server was installed and functioning properly. Good luck.

The problem: [oracle@host1 ~]$ crontab -l Authentication token is no longer valid; new one required You (oracle) are not allowed to access to (crontab) because of pam configuration. Solution: when using PAM configuration and switching between users from root with su command (or using sudo from authenticated user using auth key), when the password of the user is expired or about to be expired you may face this problem. * switch to root * change the problematic account password (using passwd command) Check yourself: * su - [account] * crontab -l good luck :-)

Hi All, This is a very simple operation intend to be very confusing as I saw in other posts I read. What you will need to add storage to hadoop is a new disk, mounted on a new volume (directory) After you got a new disk, ask you system administrator to mount him on a new desired directory: [root@RHEL7datanode1 ~]# mkdir -p /hadoop/add_disk/ [root@RHEL7datanode1 ~]# chown -R hdfs:hadoop /hadoop [root@RHEL7datanode1 ~]# su - hdfs now you need to change conf file and add this new folder to the hdfs-site.xml vi /usr/hdp/2.3.0.0-2557/hadoop/conf/hdfs-site.xml     <property>       <name>dfs.datanode.data.dir</name>       <value>/data2/hdp,/data1/hdp, /hadoop/add_disk </value>     </property> Restart datanode: [hdfs@RHEL7datanode1 ~]$ /usr/hdp/2.3.0.0-2557/hadoop/sbin/hadoop-daemon.sh stop datanode stopping datanode [hdfs@RHEL7datanode1 ~]$ /usr/hdp/2.3.0.0-2557/hadoop/sbin/hadoop-dae...

This time I faced this scenario, One of my client called me and tell me that his application server faces difficulties to create the connection pool against the DB. first I have checked the DB server side and I saw everything is ok I tried to connect from my laptop - again OK from the application server things were getting weird.. once every few tnsping commands or sqlplus command using tnsnames.ora sqlnet.ora the command tooks more than 10 seconds to return to prompt... I did these changes to isolate the problem sqlnet.ora I've changed - SQLNET . AUTHENTICATION_SERVICES = ( NTS ) to: SQLNET . AUTHENTICATION_SERVICES = ( NONE ) and  in the tnsnames.ora - I've used IP instead of the DB hostname. but still - same behavior as above. then I've started to drill down and found these amazing finding: * in the registry key called TNS_ADMIN under LOCAL MACHINE -> SOFTWARE -> ORACLE I found out that the value is pointing to a centralized Oracle client s...

Hi All, You may think that in order to query external table you should have only read permission on the Directory Object which sounds very reasonable.... but in some external table the definition include this part of code: ACCESS PARAMETERS         ( records delimited by newline           LOGFILE   "EXTERNAL_FILES" : 'faults.txt'       FIELDS TERMINATED BY ';'             missing FIELD values are null this definition force a write permission on the directory object.  else you will run into this error: ORA-29913: error in executing ODCIEXTTABLEOPEN callout ORA-29400: data cartridge error KUP-04074: no write access to directory object EXTERNAL_FILES 29913. 00000 -  "error in executing %s callout" *Cause:    The execution of the specified callout caused an error. *Acti...

if you have a process that run over your DB daily, or any other use cases that you need to recover your users after it you should simply run this query below to generate a simple script using DBMS_METADATA functions and run it over the instance when you desired to: SQL> set head off set pages 0 set long 9999999 spool create_users.sql SELECT DBMS_METADATA.GET_DDL('USER', USERNAME) || '/' DDL FROM DBA_USERS UNION ALL SELECT DBMS_METADATA.GET_GRANTED_DDL('ROLE_GRANT', USERNAME) || '/' DDL FROM  DBA_USERS UNION ALL SELECT DBMS_METADATA.GET_GRANTED_DDL('SYSTEM_GRANT', USERNAME) || '/' DDL FROM DBA_USERS UNION ALL SELECT DBMS_METADATA.GET_GRANTED_DDL('OBJECT_GRANT', USERNAME) || '/' DDL FROM DBA_USERS; spool off; enjoy :-)