Post Reply 
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
dexer
02-13-2018, 03:59 PM
Post: #31
RE: dexer
(02-13-2018 03:45 PM)rayluk Wrote:  @zma,

As from the previous discussion, I think we should let dexer return non-zero value when any of its child returned with a non-zero value.

--> http://tab.d-thinker.org/showthread.php?...http://tab.d-thinker.org/showthread.php?tid=5225&pid=6494

Not its default behavior yet. We may start with making it an explicit option to enable this behavior.

The following design looks good to me.

Quote:I suggest we can follow the approach of mulop -- use an array of PID to keep track of the situation.

Since the pseudocode on headpost is not update, following is the current pseudocode as observed
Code:
$ips?=/thinker/etc/ips.cfg
if $ips not exist then $ips=$think_base/conf/ips.cfg
if 'p'  is a parameter then pastat=True; else pastat=False
if '--shell' is a parameter then use_shell=True; else use_shell=False

if use_shell then command = bash -c "$command"

if pastat == False
  for ip in $ips
  {
           ssh -t $username@$ip "$command"
   }
else
  for ip in $ips
  {
           ssh -t $username@$ip "$command" &
   }
}


pseudocode for non zero return value
Code:
$ips?=/thinker/etc/ips.cfg
if $ips not exist then $ips=$think_base/conf/ips.cfg
if 'p'  is a parameter then pastat=True; else pastat=False
if '--shell' is a parameter then use_shell=True; else use_shell=False

if use_shell then command = bash -c "$command"


(( -> globe_rr=0 ))
if pastat == False{
  for ip in $ips
   {
           ssh -t $username@$ip "$command" && rr=$? || rr =$?
           if $rr != 0 then $globe_rr = $rr
   }
else
  (( -> pid_array="" ))
  for ip in $ips
   {
           ssh -t $username@$ip "$command" & append $! to $pid_array
   }
   for $pid in $pid_array
   {
          wait $pid && rr=$? || rr =$?
          if $rr != 0 then $globe_rr = $rr
    }
}
return $globe_rr
Visit this user's website Find all posts by this user
Quote this message in a reply
02-13-2018, 04:12 PM
Post: #32
RE: dexer
Ok, sorry for missing that post


(02-13-2018 03:59 PM)zma Wrote:  
(02-13-2018 03:45 PM)rayluk Wrote:  @zma,

As from the previous discussion, I think we should let dexer return non-zero value when any of its child returned with a non-zero value.

--> http://tab.d-thinker.org/showthread.php?...http://tab.d-thinker.org/showthread.php?tid=5225&pid=6494

Not its default behavior yet. We may start with making it an explicit option to enable this behavior.

The following design looks good to me.
Find all posts by this user
Quote this message in a reply
02-13-2018, 05:17 PM
Post: #33
RE: dexer
(02-12-2018 08:22 PM)YU_Xinjie Wrote:  I roughly remember I have proposed this issue but perhaps is still a TODO.

I have implemented a function named "wait_errexit" in cod://glad/glad/bin/glad-common.sh, which is a "return code respect" version of "wait".
You may consider to generalize/reuse that function in your design.

@rayluk

Did you see my above reply ?

Counting mulop, we are going to have 3rd duplicated code about this part. Why do not you generalize it as a general tool as I suggest ?
Find all posts by this user
Quote this message in a reply
02-13-2018, 05:48 PM
Post: #34
RE: dexer
(02-13-2018 05:17 PM)YU_Xinjie Wrote:  
(02-12-2018 08:22 PM)YU_Xinjie Wrote:  I roughly remember I have proposed this issue but perhaps is still a TODO.

I have implemented a function named "wait_errexit" in cod://glad/glad/bin/glad-common.sh, which is a "return code respect" version of "wait".
You may consider to generalize/reuse that function in your design.

@rayluk

Did you see my above reply ?

Counting mulop, we are going to have 3rd duplicated code about this part. Why do not you generalize it as a general tool as I suggest ?

I thought you want me to use the logic from "wait_errexit" in dexer. As the logic of mulop fits more in dexer, I would choose a similar approach to implement. After that, all tool can use dexer as the general tool.
Find all posts by this user
Quote this message in a reply
02-13-2018, 06:17 PM
Post: #35
RE: dexer
(02-13-2018 05:48 PM)rayluk Wrote:  I thought you want me to use the logic from "wait_errexit" in dexer. As the logic of mulop fits more in dexer, I would choose a similar approach to implement. After that, all tool can use dexer as the general tool.

I feel those are different levels things.

wait_errexit is more basic. In #29 & #33, I hoped you can improve/generalize wait_errexit and use it in dexer.

dexer is a general tool for "ssh + wait_errexit", not for wait_errexit itself.
Next time, when you use `wait` in another script, you can not use dexer to replace `wait` but you can use `wait_errexit` to replace that.
Find all posts by this user
Quote this message in a reply
02-13-2018, 06:44 PM
Post: #36
RE: dexer
(02-13-2018 06:17 PM)YU_Xinjie Wrote:  
(02-13-2018 05:48 PM)rayluk Wrote:  I thought you want me to use the logic from "wait_errexit" in dexer. As the logic of mulop fits more in dexer, I would choose a similar approach to implement. After that, all tool can use dexer as the general tool.

I feel those are different levels things.

wait_errexit is more basic. In #29 & #33, I hoped you can improve/generalize wait_errexit and use it in dexer.

dexer is a general tool for "ssh + wait_errexit", not for wait_errexit itself.
Next time, when you use `wait` in another script, you can not use dexer to replace `wait` but you can use `wait_errexit` to replace that.

But I am fine if you still would rather not generalize wait_errexit.
Find all posts by this user
Quote this message in a reply
02-15-2018, 12:04 PM (This post was last modified: 02-15-2018 12:04 PM by rayluk.)
Post: #37
RE: dexer
@Xinije, I remember I didn't use `jobs -p` in mulop as it is not reliable. When a job is done before you start `jobs -p`, it wouldn't get the pid of that job, which means it will not get the return code for that. Therefore, I will not follow the wait_errexit approach first.

Test code
Code:
[rayluk@desktop test]$ cat ./d.sh
#!/bin/bash

function pjob(){
  stime=$1
  rt=$2
  ( sleep $stime; exit $rt)&
}

echo sleep for 0 and return 1
pjob 0 1
for i in 1 2 3 4 5 6 7 8 9; do
  echo sleep for 4 and return 0
  pjob 4 0
done

echo waiting
jobpidlist=`jobs -p`
echo $jobpidlist
for pid in $jobpidlist;do
  wait $pid
  echo $?
done

After running the above code, we can observe that there are chances jobs -p misses some of the jobs when some jobs returned too fast. This is a popular case when the function failed in the first place.
Quote:[rayluk@desktop test]$ ./d.sh
sleep for 0 and return 1
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
waiting
30767 30768 30769 30771 30773 30775 30777 30778 30779 30780
1
0
0
0
0
0
0
0
0
0
[rayluk@desktop test]$ ./d.sh
sleep for 0 and return 1
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
sleep for 4 and return 0
waiting
30818 30819 30822 30823 30824 30826 30827 30828 30829
0
0
0
0
0
0
0
0
0
Find all posts by this user
Quote this message in a reply
05-30-2018, 12:06 PM (This post was last modified: 05-30-2018 12:09 PM by rayluk.)
Post: #38
RE: dexer
RR zma

Since prun is now ready, it is not suitable me to continue on the proposal in this reply

I would also like to add a new parameter --allow-error

Following is the updated proposal for using prun.

Code:
(( -> globe_rr=0 ))

if pastat == False
  ssh -t $username@$ip "$command" (( -> && rr=$? || rr =$? ))
  (( -> if $rr != 0 and globe_rr=0  then $globe_rr = $rr ))
else
  (( for ip in $ips
  {
           ssh -t $username@$ip "$command" &
  }
  ->
  array=(for ip in $ips; do put 'ssh "$usernmae@$ip" "$command" ' into the array;done)
  prun --no-error $array[@] && globe_rr =$? || globe_rr=$?
  ))
}
if --allow-error
   return globe_rr
else
   return 0
Find all posts by this user
Quote this message in a reply
05-30-2018, 12:21 PM
Post: #39
RE: dexer
(05-30-2018 12:06 PM)rayluk Wrote:  RR zma

Since prun is now ready, it is not suitable me to continue on the proposal in this reply

I would also like to add a new parameter --allow-error

Following is the updated proposal for using prun.

Code:
(( -> globe_rr=0 ))

if pastat == False
  ssh -t $username@$ip "$command" (( -> && rr=$? || rr =$? ))
  (( -> if $rr != 0 and globe_rr=0  then $globe_rr = $rr ))
else
  (( for ip in $ips
  {
           ssh -t $username@$ip "$command" &
  }
  ->
  array=(for ip in $ips; do put 'ssh "$usernmae@$ip" "$command" ' into the array;done)
  prun --no-error $array[@] && globe_rr =$? || globe_rr=$?
  ))
}
if --allow-error
   return globe_rr
else
   return 0

OK except --allow-error. What's the semantics of it?
Visit this user's website Find all posts by this user
Quote this message in a reply
05-30-2018, 12:31 PM
Post: #40
RE: dexer
(05-30-2018 12:21 PM)zma Wrote:  
(05-30-2018 12:06 PM)rayluk Wrote:  RR zma

Since prun is now ready, it is not suitable me to continue on the proposal in this reply

I would also like to add a new parameter --allow-error

Following is the updated proposal for using prun.

Code:
(( -> globe_rr=0 ))

if pastat == False
  ssh -t $username@$ip "$command" (( -> && rr=$? || rr =$? ))
  (( -> if $rr != 0 and globe_rr=0  then $globe_rr = $rr ))
else
  (( for ip in $ips
  {
           ssh -t $username@$ip "$command" &
  }
  ->
  array=(for ip in $ips; do put 'ssh "$usernmae@$ip" "$command" ' into the array;done)
  prun --no-error $array[@] && globe_rr =$? || globe_rr=$?
  ))
}
if --allow-error
   return globe_rr
else
   return 0

OK except --allow-error. What's the semantics of it?

--allow-error controlls the return value fo dexer
when --allow-error is not specified, it follows the old approach. It returns 0 no matter anything failed.
when --allow-error is specified, it would return the first non-zero return value of subprocesses. If all subprocesses returns zero, dexer will also returns zero.
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump: