Nsryjdtyk

CREATE TABLE `cluster_diagnostic_report`(

  `run_id` string COMMENT 'format: <hostname>_<datetime> - to uniquely identify the a particular execution instance of Cluster Diag job',

  `execution_hostname` string COMMENT 'Machine Name from where Test Case Executed',

  `module` string COMMENT 'Test Case Module',

  `expected_result` string COMMENT 'Test Case Module expected Result',

  `actual_result` string COMMENT 'Test Case Module actual Result',

  `validation_result` string COMMENT 'Test Case Module validation Result',

  `start_time` string COMMENT 'Test Case Module Start Time',

  `end_time` string COMMENT 'Test Case Module Elapsed Time',

  `elapsed_time` string COMMENT 'from deserializer',

  `total_time_seconds` int COMMENT 'total elapsed time for this step')

PARTITIONED BY (

  `cluster_name` string,

  `rptg_dt` string)

ROW FORMAT SERDE

  'org.apache.hadoop.hive.ql.io.orc.OrcSerde'

from the above contect i nees to get only the partitioned column name and type.
for the above example i want to get details as follows :

col_name = cluster_name, type = string

rptg_dt= cluster_name, type = string

what i have tried is given below buyt it is returning None:

partitionResult = re.match(r"PARTITIONEDsw+s((n){2}",line)

if partitionResult == None:

    pass

else:

    print(partitionResult.group(1),sep='t')

can anyone please suggest what to do?

Here is a solution that uses G (continue from the start or previous match) to match an arbitrary number of cluster cols/types:

Online Test (needs to run in PCRE)

Sample Code (requires alternative regex package for Python)

import regex as re



regex = r"(?|PARTITIONEDs+BYs+(s+`(w+)`s+(w+),?|Gs*`(w+)`s+(w+),?)K"



test_str = ("CREATE TABLE `cluster_diagnostic_report`(n"

    "  `run_id` string COMMENT 'format: <hostname>_<datetime> - to uniquely identify the a particular execution instance of Cluster Diag job',n"

    "  `execution_hostname` string COMMENT 'Machine Name from where Test Case Executed',n"

    "  `module` string COMMENT 'Test Case Module',n"

    "  `expected_result` string COMMENT 'Test Case Module expected Result',n"

    "  `actual_result` string COMMENT 'Test Case Module actual Result',n"

    "  `validation_result` string COMMENT 'Test Case Module validation Result',n"

    "  `start_time` string COMMENT 'Test Case Module Start Time',n"

    "  `end_time` string COMMENT 'Test Case Module Elapsed Time',n"

    "  `elapsed_time` string COMMENT 'from deserializer',n"

    "  `total_time_seconds` int COMMENT 'total elapsed time for this step')n"

    "PARTITIONED BY (n"

    "  `cluster_name` string,n"

    "   `cluster_name2` string,`rptg_dt` string,n"

    "`cluster_name2` string,)n"

    "ROW FORMAT SERDEn"

    "  'org.apache.hadoop.hive.ql.io.orc.OrcSerde'")



matches = re.finditer(regex, test_str, re.MULTILINE)



for matchNum, match in enumerate(matches):    

    for groupNum in range(0, len(match.groups())):

        groupNum = groupNum + 1



        print ("Group {groupNum} found at {start}-{end}: {group}".format(groupNum = groupNum, start = match.start(groupNum), end = match.end(groupNum), group = match.group(groupNum)))

Output:

Group 1 found at 789-801: cluster_name

Group 2 found at 803-809: string

Group 1 found at 813-826: cluster_name2

Group 2 found at 828-834: string

Group 1 found at 836-843: rptg_dt

Group 2 found at 845-851: string

Group 1 found at 854-867: cluster_name2

Group 2 found at 869-875: string

Make use of positive look ahead and look behind (Example):

pat = re.compile(r'(?<=PARTITIONED BY (n)(.*)(?=))', re.S)

# Look behind for "PARTITIONED BY (" and look ahead to ")"

# use the re.S flag to match across lines with .*



results = pat.search(text).group()



# '  `cluster_name` string,n  `rptg_dt` string'



# ... do what you need with the strings.



[i.lstrip() for i in results.split('n')]

# ['`cluster_name` string,', 

#  '`rptg_dt` string']