Stosujemy obejście:
alter table TABELA enable row movement;
robimy update-a (to co nie poszło wcześniej) i wyłączamy:
alter table TABELA disable row
…
Autor: Wiech
ORA-00600: internal error code, arguments: [krccacp_badfile], (CTWR)), summary=[abnormal instance termination].
błąd:
Fri Jan 28 14:13:11 2022
ERROR: Unable to normalize symbol name for the following short stack (at offset 204):
…
ORA-00600: internal error code, arguments: [krccacp_badfile], [6964816530811], [6956618400402], [], [], [], [], [], [], [], [], []
Incident details in: /u01/diag/rdbms/baza/baza/incident/incdir_32483/baza_ctwr_14549066_i32483.trc
…
Fri Jan 28 14:13:13 2022
Dumping diagnostic data in directory=[cdmp_20220128141313], requested by (instance=1, osid=14549066 (CTWR)), summary=[incident=32483].
…
Fri Jan 28 14:13:25 2022
Errors in file /u01/diag/rdbms/baza/baza/trace/baza_ctwr_14549066.trc:
ORA-00600: internal error code, arguments: [krccacp_badfile], [6964816530811], [6956618400402], [], [], [], [], [], [], [], [], []
CTWR (ospid: 14549066): terminating the instance due to error 487
Fri Jan 28 14:13:25 2022
System state dump requested by (instance=1, osid=14549066 (CTWR)), summary=[abnormal instance termination].
System State dumped to trace file /u01/diag/rdbms/baza/baza/trace/baza_diag_22282266.trc
Dumping diagnostic data in directory=[cdmp_20220128141325], requested by (instance=1, osid=14549066 (CTWR)), summary=[abnormal instance termination].
Instance terminated by CTWR, pid = 14549066
W przypadku, gdy odtwarzamy bazę na tej samej maszynie jako kolejną i pomimo tego, że mamy zmieniony dbid, nazwe itp. jedna baza w danym czasie może być dostępna, bo mamy włączony block change tracking (odnosi się do tego samego pliku), sprawdzamy na 2ch bazach:
…
[FAIL] *** dm02dbadm01: Stale map files found in /etc/oracle/maps. You may need to remove the stale map files to avoid the cluster start up issues – Refer the confluence page for more details
— sprawdzamy kiedy byl ostatni boot crs-a (na wszystkich nodach)
cat /u01/app/grid/diag/crs/dm02dbadm0*/crs/trace/alert.log | grep 'CRS-1301′ | tail -n 1 | awk '{print $1,$2,$3}’
— szukamy plików z ostatnim accessem wczesniejszym niz data wyzej czyli boot-a crs-a (mozna find z atime, albo stat)
[root@dm02dbadm01 maps]# ls -latr –time-style=long /etc/oracle/maps/
total 44
-rwxrw—- 1 grid oinstall 0 2020-10-20 19:28 gipcds_gipc_diag_b026c96e3249ff41ff85063dfaaa7e0e_dm02dbadm01_lock
-rw-r—– 1 grid oinstall 66304 2020-10-21 06:14 gipcds_gipc_diag_b026c96e3249ff41ff85063dfaaa7e0e_dm02dbadm01-0000000000
-rwxrw—- 1 grid oinstall 0 2021-01-29 00:25 gipcDS_dm02dbadm01_gipcd_b026c96e3249ff41ff8
…
[FAIL] *** dm02dbadm01: HugePages does not match value in sysctl.conf. This may prevent system boot. (found: 46581, sysctl.conf: 46077)
— sprawdzamy jak to wygląda:
[user@dm02dbadm01 ~]$ cat /etc/sysctl.conf | grep -A1 -B1 vm.nr_hugepages
##### End: Generated by Oracle Exadata ipconf
#vm.nr_hugepages=13827
vm.nr_hugepages=46077
##### BEGIN DO NOT REMOVE Added by Oracle Exadata
[user@dm02dbadm01 ~]$ grep Huge /proc/meminfo
AnonHugePages: 0 kB
ShmemHugePages: 0 kB
HugePages_Total: 46581
HugePages_Free: 20272
HugePages_Rsvd: 11160
HugePages_Surp: 0
Hugepagesize: 2048 kB
— w err/zaleceniu jak i w pliku mamy poprawną wartość czyli: 46077, ale w pamieci jest wartość większa (gdy zmieniamy na wartość mniejszą i HugePages_Free zje całe, to bezpieczniej jest polożyc crs-a)
[root@dm02dbadm01 ~]# sysctl -p
— sprawdzamy
…
SRL log 1 needs clearing because log has not been created
Po aplikowaniu incrementala (DG move forward: https://ora-600.com/2021/03/err-replikacji-danych-data-guard-move-forward-czyli-aplikowanie-intrementala/) możemy trafić na:
SRL log 1 needs clearing because log has not been created
SRL log 2 needs clearing because log has not been created
SRL log 3 needs clearing because log has not been created
SRL log 4 needs clearing because log has not been created
Zatrzymujemy aplikowanie archow, czyścimy grupy logów i odpalamy:
alter database recover managed standby database cancel;
alter database clear logfile group 1;
alter database clear logfile group 2;
alter database clear logfile group 3;
…
DLL – brak biblioteki (gdzie ją wcisnąć)
Jeśli brakuje jakiejś dll-ki tak jak tu orannzsbb12.dll i nie wiemy gdzie ją wcisnąć, to odpalamy Process Monitor (@sysinternals) i od razu widać gdzie powinna być (patrz screen):
16:53:27,6562347 impdp.exe 4048 CreateFile C:\Windows\System32\orannzsbb12.dll NAME NOT FOUND Desired Access: Read Attributes, Dispositi
…
ORA-29701: unable to connect to Cluster Synchronization Service, Error 29701
Pacjent: Grid 19.13, DB 12.2.0.1, OL 7.9 – NODE2 łapie freeza po starcie i trzeba pocisnąć go z palca (po błędach związanych z interconnectem w logu CRS-a)
— w alercie DB NODE1
2021-11-04T14:26:11.977078+01:00
JIT: pid 44720 requesting full stop
2021-11-04T14:26:18.243731+01:00
JIT: pid 44720 requesting full stop
2021-11-04T14:33:53.984937+01:00
IPC Send timeout detected. Sender: ospid 20158 [oracle@NODE1 (LCK0)]
Receiver: inst 2 binc 16 ospid 18523
2021-11-04T14:33:53.994876+01:00
Communications reconfiguration: instance_number 2 by ospid 20158
2021-11-04T14:34:42.795807+01:00
Detected an inconsistent instance membership by instance 1
Evicting instance 2 from cluster
Waiting for instances to leave: 2
2021-11-04T14:34:42.950298+01:00
IPC Send timeout to 2.1 inc 20 for msg type 65521 from opid 24
2021-11-04T14:34:42.950358+01:00
IPC Send timeout to 2.1 inc 20 for msg type 65521 from opid 24
— a w logu CRS-a NODE2 (na NODE1 są jedynie informacje o problemach z NODE2 bez countdownu):
2021-11-04 14:37:21.717 [OCSSD(10029)]CRS-7503: The Oracle Grid Infrastructure process ocssd observed communication issues between node NODE2 and node NODE1, interface list of local node NODE2 is 172.30.1.2:20313, interface list of remote node NODE1 is 172.30.1.1:64128.
2021-11-04 14:37:27.240 [OCSSD(10029)]CRS-1612: Network communication with node NODE1 (1) has been missing for 50% of the timeout interval. If this persists, removal of this no
…
ORA-02062: distributed recovery received DBID xxx, expected xxy – wycofanie transakcji
błąd:
Errors in file /u01/diag/rdbms/baza/baza/trace/baza_reco_20054068.trc:
ORA-02062: distributed recovery received DBID xxx, expected xxy
— sprawdzenie oczekujących transakcji
SELECT LOCAL_TRAN_ID, GLOBAL_TRAN_ID, STATE, MIXED, COMMIT# FROM DBA_2PC_PENDING;
— jezeli status prepaired to wycofujemy, albo commitujemy (LOCAL_TRAN_ID):
ROLLBACK FORCE '107.16.2563092′;
COMMIT FORCE '107.16.2563092′;
— albo:
DELETE FROM SYS.PENDING_TRANS$ WHE
…
Suspending MMON slave action kewrmafsa_ for 82800 seconds
Suspending MMON slave action kewrmafsa_ for 82800 seconds
— ORA pod większym obciążeniem stwierdził, że MMON jest mniej istotny i go uspił
— jeżeli potrzebujemy na teraz SNAP-a (AWR-a) to wymuszamy:
EXEC DBMS_WORKLOAD_REPOSITORY.create_snapshot;
— sprawdzenie snapow:
SELECT * FROM (SELECT instance_number, startup_time, begin_interval
…
clnt_create: RPC: Port mapper failure – Unable to receive: errno 113 (No route to host)
# edytujemy i dorzucamy po stronie serva:
vi /etc/sysconfig/nfs
MOUNTD_PORT=”888″
STATD_PORT=”889″
RQUOTAD_PORT=”890″
LOCKD_TCPPORT=”891″
LOCKD_UDPPORT=”891″
# FW
iptables -I INPUT -m state –state NEW -p tcp –dport 888 -j ACCEPT
iptables -I INPUT -m state –state NEW -p udp –dport 888 -j ACCEPT
iptables -I INPUT -m state –state NEW -p tcp –dport 889 -j ACCEPT
iptables -I INPUT -m state –state NEW -p udp –dport 889 -j ACCEPT
iptables -I INPUT -m state –state NEW -p tcp –dport 891 -j ACCEPT
iptables -I INPUT -m state –state NEW -p udp –dport 891 -j ACCEPT
iptables -I INPUT -m state –state NEW -p tcp –dpo
…