Post Reply 
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
AID Usage
01-31-2018, 02:50 PM (This post was last modified: 01-31-2018 02:51 PM by rayluk.)
Post: #1
AID Usage
When in customer's site, as AID is a storage machine, we will have to let the customers to connect to the storage system.

Possible solutions:

# on client's linux mahcine
1) mount by ceph kernel client
Highly not recommended as we seldom know what our client's kernel version. We have experienced bug on old ceph kernel client and some old kernel may not work with our version of ceph. ( old means the 4.9 kernel or before )

2) mount by NFS
We can use NFS to export ceph. and let clients to connect to the NFS. Ceph would be acting as the underlying storage system.

3) mount by fuse client
This can be an option if the client's machine is in the same network of ceph. But I wouldn't suggest this as our GM seldom share same network with client's network. It would be better for AID to follow the same practise.

# on client's windows machine
1) mount by samba
I think this is the best and only option. ceph would be acting like the underlying file system.
Find all posts by this user
Quote this message in a reply
01-31-2018, 09:17 PM
Post: #2
RE: AID Usage
(01-31-2018 02:50 PM)rayluk Wrote:  When in customer's site, as AID is a storage machine, we will have to let the customers to connect to the storage system.

Possible solutions:

# on client's linux mahcine
1) mount by ceph kernel client
Highly not recommended as we seldom know what our client's kernel version. We have experienced bug on old ceph kernel client and some old kernel may not work with our version of ceph. ( old means the 4.9 kernel or before )

2) mount by NFS
We can use NFS to export ceph. and let clients to connect to the NFS. Ceph would be acting as the underlying storage system.

3) mount by fuse client
This can be an option if the client's machine is in the same network of ceph. But I wouldn't suggest this as our GM seldom share same network with client's network. It would be better for AID to follow the same practise.

# on client's windows machine
1) mount by samba
I think this is the best and only option. ceph would be acting like the underlying file system.

Not sure of samba's perf.

Can we provide both NFS and samba?

Also, can we provide several NFS service points so that 100 clients can connect to 5 NFS servers?
Find all posts by this user
Quote this message in a reply
02-01-2018, 10:50 AM
Post: #3
RE: AID Usage
(01-31-2018 09:17 PM)lingu Wrote:  
(01-31-2018 02:50 PM)rayluk Wrote:  When in customer's site, as AID is a storage machine, we will have to let the customers to connect to the storage system.

Possible solutions:

# on client's linux mahcine
1) mount by ceph kernel client
Highly not recommended as we seldom know what our client's kernel version. We have experienced bug on old ceph kernel client and some old kernel may not work with our version of ceph. ( old means the 4.9 kernel or before )

2) mount by NFS
We can use NFS to export ceph. and let clients to connect to the NFS. Ceph would be acting as the underlying storage system.

3) mount by fuse client
This can be an option if the client's machine is in the same network of ceph. But I wouldn't suggest this as our GM seldom share same network with client's network. It would be better for AID to follow the same practise.

# on client's windows machine
1) mount by samba
I think this is the best and only option. ceph would be acting like the underlying file system.

Not sure of samba's perf.

I would put samba's perf as a test to be done.

I am not sure if we have window machine to test the perf but I think we can mount it to test on linux.

(01-31-2018 09:17 PM)lingu Wrote:  Can we provide both NFS and samba?

I think both NFS and samba would be fine but need to be tested first. There may be some troubles regarding but that may be handled by kernel already.

Quote:Also, can we provide several NFS service points so that 100 clients can connect to 5 NFS servers?

I think yes if we are not going to use HDFS in AID as the storage system. Each VM can be a NFS server. But I think we should start from 1 in the design first and increase the amount when we see there are needs.

Performance of multiple NFS will also be a test
Find all posts by this user
Quote this message in a reply
02-01-2018, 11:46 AM
Post: #4
RE: AID Usage
(02-01-2018 10:50 AM)rayluk Wrote:  
(01-31-2018 09:17 PM)lingu Wrote:  
(01-31-2018 02:50 PM)rayluk Wrote:  When in customer's site, as AID is a storage machine, we will have to let the customers to connect to the storage system.

Possible solutions:

# on client's linux mahcine
1) mount by ceph kernel client
Highly not recommended as we seldom know what our client's kernel version. We have experienced bug on old ceph kernel client and some old kernel may not work with our version of ceph. ( old means the 4.9 kernel or before )

2) mount by NFS
We can use NFS to export ceph. and let clients to connect to the NFS. Ceph would be acting as the underlying storage system.

3) mount by fuse client
This can be an option if the client's machine is in the same network of ceph. But I wouldn't suggest this as our GM seldom share same network with client's network. It would be better for AID to follow the same practise.

# on client's windows machine
1) mount by samba
I think this is the best and only option. ceph would be acting like the underlying file system.

Not sure of samba's perf.

I would put samba's perf as a test to be done.

I am not sure if we have window machine to test the perf but I think we can mount it to test on linux.

(01-31-2018 09:17 PM)lingu Wrote:  Can we provide both NFS and samba?

I think both NFS and samba would be fine but need to be tested first. There may be some troubles regarding but that may be handled by kernel already.

Quote:Also, can we provide several NFS service points so that 100 clients can connect to 5 NFS servers?

I think yes if we are not going to use HDFS in AID as the storage system. Each VM can be a NFS server. But I think we should start from 1 in the design first and increase the amount when we see there are needs.

Performance of multiple NFS will also be a test

Some comments:

1. Samba server daemon and NFS server daemon can run on top of the same backend filesystem on the same server.

2. Samba performance is expected not as good as NFS. It's performance may be just reasonable so that Windows nodes can use it. I don't expect building high performance FS connector on top of Samba.
But I don't suggest making it enabled by default.

Windows users can use WinSCP which is securer and can be used in wider cases. From our customers, they are happy with WinSCP.

Only for some wired cases like sequencers that supports only Samba, we may enable Samba as an additional module.

3. Multiple NFS on the same storage should be fine. But the actual consistency model may not be expected by all programs. We are doing so for D-store/HDFS. I agree with Ray to start with one first so that AID can start to work quickly.
Quote this message in a reply
02-01-2018, 01:04 PM
Post: #5
RE: AID Usage
(02-01-2018 10:50 AM)rayluk Wrote:  I think yes if we are not going to use HDFS in AID as the storage system. Each VM can be a NFS server. But I think we should start from 1 in the design first and increase the amount when we see there are needs.

Performance of multiple NFS will also be a test

Start from 2 NFS service points. This can expose problems. If there is serious problems, we may have to cancel the plan of using Ceph.

A big part of the art of designing a large system is to handle uncertainty. Remove big uncertainties as early as possible.
Find all posts by this user
Quote this message in a reply
02-01-2018, 01:08 PM (This post was last modified: 02-01-2018 01:08 PM by lingu.)
Post: #6
RE: AID Usage
(02-01-2018 11:46 AM)zma Wrote:  
(02-01-2018 10:50 AM)rayluk Wrote:  
(01-31-2018 09:17 PM)lingu Wrote:  I would put samba's perf as a test to be done.

I am not sure if we have window machine to test the perf but I think we can mount it to test on linux.

[quote='lingu' pid='63814' dateline='1517404622']
Can we provide both NFS and samba?

I think both NFS and samba would be fine but need to be tested first. There may be some troubles regarding but that may be handled by kernel already.

Quote:Also, can we provide several NFS service points so that 100 clients can connect to 5 NFS servers?

I think yes if we are not going to use HDFS in AID as the storage system. Each VM can be a NFS server. But I think we should start from 1 in the design first and increase the amount when we see there are needs.

Performance of multiple NFS will also be a test

Some comments:

1. Samba server daemon and NFS server daemon can run on top of the same backend filesystem on the same server.

2. Samba performance is expected not as good as NFS. It's performance may be just reasonable so that Windows nodes can use it. I don't expect building high performance FS connector on top of Samba.
But I don't suggest making it enabled by default.

Windows users can use WinSCP which is securer and can be used in wider cases. From our customers, they are happy with WinSCP.

Only for some wired cases like sequencers that supports only Samba, we may enable Samba as an additional module.

3. Multiple NFS on the same storage should be fine. But the actual consistency model may not be expected by all programs. We are doing so for D-store/HDFS. I agree with Ray to start with one first so that AID can start to work quickly.

Dont take a technical view. Technology is just 5% of a tech company.

Pls think about what the customers want, what product we are to ship, and what the market is.

AID should support all kinds of network FS interfaces. Otherwise, its competence on the market is much weakened.

Multiple-NFS consistency is a big issue. Pls clarify what consistency it is. We need think about it hard. In the worst case, we have to re-define AID or even abandon it.
Find all posts by this user
Quote this message in a reply
02-12-2018, 05:12 PM
Post: #7
RE: AID Usage
As discussed with zma and lingu in the afternoon,

The orientation of AID can be
- a NAS that can be easily extended ( we will have to smooth the disk replacement process )
- a single node of computation and storage that supports a small sequencer ( with glad and gladseq on it )

Sales
- may start from existing customer. But they may not have large needs on it as they already got a large storage cluster.

Others
- rename AID may be needed as it is easy to be confused with RAID
Find all posts by this user
Quote this message in a reply
02-14-2018, 04:46 PM
Post: #8
RE: AID Usage
@zma,

About the network services, after some thinking, I suggest we do it on PH.
The reason VM cannot be directly cannot from the clients network most of the case. If we want to use VM to host tNFS and samba, we will have to do port-forwarding on the PH, which may cause some other troubles.

Based on the above proposal, we will have to make
M1) yotta mounting only but with no storage device ( on the PH)
M2) path based tNFS for pointing to the directory of ceph on the PH
M3) samba module and path based samba.
Find all posts by this user
Quote this message in a reply
02-14-2018, 04:53 PM
Post: #9
RE: AID Usage
(02-14-2018 04:46 PM)rayluk Wrote:  @zma,

About the network services, after some thinking, I suggest we do it on PH.
The reason VM cannot be directly cannot from the clients network most of the case. If we want to use VM to host tNFS and samba, we will have to do port-forwarding on the PH, which may cause some other troubles.

Based on the above proposal, we will have to make
M1) yotta mounting only but with no storage device ( on the PH)
M2) path based tNFS for pointing to the directory of ceph on the PH
M3) samba module and path based samba.

The solution sounds more complex.

On the PH site, a lot of work should be done. Port forwarding is more lightweight. Most work/modules are in VMs' site. On PH gateway, only port forwarding settings are needed. The VM site is very close to a GM/TB's site.

tNFS samba may be hosted on different VMs. PH based solution will make this impossible if there is only one PH.
Quote this message in a reply
02-14-2018, 05:00 PM
Post: #10
RE: AID Usage
(02-14-2018 04:53 PM)zma Wrote:  
(02-14-2018 04:46 PM)rayluk Wrote:  @zma,

About the network services, after some thinking, I suggest we do it on PH.
The reason VM cannot be directly cannot from the clients network most of the case. If we want to use VM to host tNFS and samba, we will have to do port-forwarding on the PH, which may cause some other troubles.

Based on the above proposal, we will have to make
M1) yotta mounting only but with no storage device ( on the PH)
M2) path based tNFS for pointing to the directory of ceph on the PH
M3) samba module and path based samba.

The solution sounds more complex.

On the PH site, a lot of work should be done. Port forwarding is more lightweight. Most work/modules are in VMs' site. On PH gateway, only port forwarding settings are needed. The VM site is very close to a GM/TB's site.

tNFS samba may be hosted on different VMs. PH based solution will make this impossible if there is only one PH.

Ok, in such case, we will still need M2 and M3 . While I suggest we let post-autation supports port forwarding. I will try to find the needed ports for samba and tNFS.
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump: